Computer system and method of presenting information on basis of prediction result for input data

ABSTRACT

A computer system stores interpretation factor conversion information for managing an interpretation factor interpreting a basis of a prediction result for input data, the interpretation factor is determined by a value of each of a plurality of feature quantities contained in the input data including values of the plurality of feature quantities, and a first evaluation value of each of the plurality of feature quantities contained in the input data. When evaluation target data is input, the computer system calculates a prediction result, calculates a contribution value of each of the plurality feature quantities contained in the evaluation target data, specifies a corresponding interpretation factor, based on a value and a contribution value of each of the plurality of feature quantities contained in the evaluation target data, by referring to the interpretation factor conversion information, and generates and outputs display information for presenting the specified interpretation factor.

CLAIM OF PRIORITY

The present application claims priority from Japanese patent applicationJP 2019-014860 filed on Jan. 30, 2019, the content of which is herebyincorporated by reference into this application.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a system and method of presentinguseful data for interpreting a basis for a prediction result of AI thatperforms a prediction based on a model constructed by machine learning.

2. Description of the Related Art

In recent years, an assistance system that utilizes AI in various fieldssuch as a medical field and a financial field has been provided. Forexample, in the medical field, the AI is utilized to predict a diseaseincidence rate (event) of a person (subject) and to specify symptoms. Inthe financial field, the credit review or the like is performed byutilizing the AI. For example, an occurrence rate (event) of loan lossof an applicant (subject) is predicted.

As a result of advances in technological development aimed at improvingaccuracy of prediction results output by the AI, such as prediction ofdisease incidence, the black-box processing of an AI model (algorithm)is accelerated. Therefore, the user who uses the AI cannot trust thepredicted value (prediction result) of the AI.

Under such a background, there is an increasing demand for describing abasis of the prediction result of AI and verifying an operation by adeveloper or an operator of a system utilizing AI.

JP-A-2018-147280 and Lundberg, Scott M., and Su-In Lee. “A unifiedapproach to interpreting model predictions.” Advances in NeuralInformation Processing Systems (pp. 4765-4774). 2017. (Non-PatentLiterature 1) provides techniques for presenting information indicatingreliability of a system to a user who uses the system.

JP-A-2018-147280 states that “A data analysis device includes a data setgeneration unit that generates explanatory variable data to be input toa machine learning model and inputs the generated explanatory variabledata to the machine learning model to obtain objective variable data;and a model evaluation unit that calculates a relationship between theexplanatory variable and the objective variable based on the explanatoryvariable data and the objective variable data generated by the data setgeneration unit.”.

JP-A-2017-123088 states that “A contribution degree for each explanatoryvariable is calculated by a decision tree learning algorithm, and groupseach having a first predetermined number of first explanatory variablesare extracted from a higher rank in descending order of contributiondegree.”

Non-Patent Literature 1 describes a method of calculating a contributiondegree to a predicted value of AI for each feature quantity ofevaluation target data by using a combination of a plurality of piecesof intermediate evaluation data generated from the evaluation targetdata and comparison target data, and predicted values obtained byinputting each perturbation data to the AI.

The relationship between the explanatory variable and the objectivevariable calculated by the data analysis method in JP-A-2018-147280 andthe data calculated in Non-Patent Literature 1 are influence degree onthe objective variable for each explanatory variable. However, even ifthe explanatory variable name and the contribution degree are presentedto an end user, the end user hardly understands and interprets themeaning thereof and cannot confirm authenticity of the prediction resultby collating job knowledge of the user.

In the technique described in JP-A-2017-123088, the explanatory variablegroup is extracted in descending order of the contribution degreewithout using job knowledge possessed by the user. Therefore, theextracted explanatory variable group cannot be interpreted from theviewpoint of the job knowledge of the user.

SUMMARY OF THE INVENTION

The present invention provides a system and method of presentinginformation useful for a user to interpret a basis of a predictionresult output by AI.

A representative example of the invention disclosed in the presentapplication is as follows. That is, the invention provides a computersystem including at least one computer that includes a processor and amemory connected to the processor, in which the memory stores modelinformation for predicting an event of a target based on input dataincluding a value of a plurality of feature quantities indicating astate of the target, and interpretation factor conversion informationfor managing an interpretation factor interpreting a basis of aprediction result for the input data, the interpretation factor beingdetermined by a value of each of the plurality of feature quantitiescontained in the input data and a first evaluation value of the value ofeach of the plurality of feature quantities contained in the input data,the first evaluation value showing magnitude of contribution of thevalue of the feature quantity to a prediction result for the input data,and the processor is configured to calculate, when evaluation targetdata which is the input data to be evaluated is input, a predictionresult for the evaluation target data based on the model information,calculate a first evaluation value of each of the plurality of featurequantities contained in the evaluation target data, specify acorresponding interpretation factor, based on the value and firstevaluation value of each of the plurality of feature quantitiescontained in the evaluation target data, by referring to theinterpretation factor conversion information, and generate displayinformation for presenting the specified interpretation factor andoutput the display information.

According to the aspect of the present invention, data useful for theuser to interpret the basis of the prediction result output from AI canbe presented. Problems, configurations, and effects other than thosedescribed above will be apparent from the following description of theembodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of a configuration of a computer systemaccording to a first embodiment.

FIG. 2 shows an example of a hardware configuration of a computeraccording to the first embodiment.

FIG. 3 shows an example of a data structure of evaluation target dataand output data according to the first embodiment.

FIG. 4 shows an example of a data structure of management information ofcomparison target data according to the first embodiment.

FIG. 5 shows an example of a data structure of interpretation factorconversion information according to the first embodiment.

FIG. 6 shows a flow of processing of the computer system according tothe first embodiment.

FIG. 7 shows an example of a data structure of contribution value dataaccording to the first embodiment.

FIG. 8 shows an example of a data structure of interpretationinformation according to the first embodiment.

FIG. 9A shows an example of an input screen displayed on a terminalaccording to the first embodiment.

FIG. 9B shows an example of an input screen displayed on the terminalaccording to the first embodiment.

FIG. 10 is a flowchart showing an example of contribution value datageneration processing executed by a contribution value calculation unitaccording to the first embodiment.

FIG. 11A shows an example of a data structure of intermediate dataaccording to the first embodiment.

FIG. 11B shows an example of a data structure of intermediate dataaccording to the first embodiment.

FIG. 11C shows an example of a data structure of intermediate dataaccording to the first embodiment.

FIG. 11D shows an example of a data structure of intermediate dataaccording to the first embodiment.

FIG. 11E shows an example of a data structure of intermediate dataaccording to the first embodiment.

FIG. 12 is a flowchart showing an example of interpretation informationgeneration processing executed by an interpretation factor selectionunit according to the first embodiment.

FIG. 13 shows an example of a result output screen displayed on theterminal according to the first embodiment.

FIG. 14 shows a configuration example of a computer system according toa second embodiment.

FIG. 15 shows a flow of processing of the computer system according tothe second embodiment.

FIG. 16 shows an example of a data structure of data for generationaccording to the second embodiment.

FIG. 17 shows an example of a data structure of intermediate calculationdata according to the second embodiment.

FIG. 18 shows an example of a data structure of area divisioninformation according to the second embodiment.

FIG. 19 shows an example of a data structure of candidate interpretationfactor information according to the second embodiment.

FIG. 20 shows an example of a data structure of a candidate conversiontable according to the second embodiment.

FIG. 21 shows an example of a setting screen displayed on a terminalaccording to the second embodiment.

FIG. 22 shows an example of a division condition detail setting screendisplayed on the terminal according to the second embodiment.

FIG. 23 shows an example of a confirmation screen displayed on theterminal according to the second embodiment.

FIG. 24A is a flowchart showing an example of processing executed by acandidate interpretation factor generation unit according to the secondembodiment.

FIG. 24B is a flowchart showing the example of the processing executedby the candidate interpretation factor generation unit according to thesecond embodiment.

FIG. 25 shows an example of a data structure of an interaction valuelist according to the second embodiment.

FIG. 26 shows processing executed by a conversion table generation unitaccording to the second embodiment.

FIG. 27 shows an example of a data structure of candidate interpretationfactor information according to a third embodiment.

FIG. 28A is a flowchart showing an example of processing executed by acandidate interpretation factor generation unit according to the thirdembodiment.

FIG. 28B is a flowchart showing the example of the processing executedby the candidate interpretation factor generation unit according to thethird embodiment.

FIG. 29 shows an example of an integration condition setting screendisplayed on a terminal according to the third embodiment.

FIG. 30 shows an example of an integration confirmation screen displayedon the terminal according to the third embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of the invention will be described withreference to the drawings. However, these embodiments are merelyexamples for implementing the invention, and do not limit the technicalscope of the invention. It will be readily understood by those skilledin the art that the specific configuration may be modified withoutdeparting from the spirit or scope of the invention.

In configurations according to the invention described below, the sameor similar configurations or functions are denoted by the same referencenumerals, and repeated descriptions thereof are omitted.

Descriptions such as “first”, “second”, and “third” in the presentspecification are used to identify constituent elements and do notnecessarily limit the number or the order.

In order to facilitate understanding of the invention, the position,size, shape, range, and the like of each component shown in the drawingsmay not represent actual position, size, shape, range, and the like.Therefore, the invention is not limited to the position, size, shape,and range disclosed in the drawings.

First Embodiment

FIG. 1 shows an example of a configuration of a computer systemaccording to a first embodiment.

The computer system includes a plurality of computers 100-1, 100-2, and100-3, and a terminal 101. The plurality of computers 100-1, 100-2, and100-3 and the terminal 101 are connected to each other via a network102. The network 102 is, for example, a wide area network (WAN) or alocal area network (LAN). A connection method of the network 102 may beeither wired or wireless.

In the following description, the computers 100-1, 100-2, and 100-3 aredescribed as a computer 100 when not being distinguished from oneanother.

The terminal 101 is a computer operated by a user. The terminal 101 is,for example, a personal computer, a smart phone, a tablet terminal, orthe like. The terminal 101 inputs, based on an operation of a user,input data and the like necessary for prediction of a target event byAI, and the like. The input data includes values of a plurality offeature quantities. Evaluation target data 300 (see FIG. 3),intermediate data 1100 (see FIG. 11), and data for generation 1500 (seeFIG. 15) are examples of the input data.

The terminal 101 includes a processor, a memory, a network interface, aninput device, and an output device. The input device is a device such asa keyboard, a mouse, and a touch panel, and the output device is adevice such as a touch panel and a display.

The computer 100-1 executes processing for the input data based on amodel (algorithm) for predicting an event of a target, and outputs apredicted value (prediction result). The output predicted value is, forexample, a classification result of the input data and an occurrenceprobability of any risk. The computer 100-1 includes a predictor 110.

The predictor 110 performs processing for the input data based on modelinformation defining the model, and outputs a predicted value. The modelinformation is generated by machine learning or the like using learningdata. Information on a neural network or a decision tree is stored inthe model information.

The computer 100-2 calculates, for a value of each feature quantitycontained in the input data, a contribution value (a first evaluationvalue) which represents magnitude of contribution (magnitude ofinfluence) to a predicted value of a value of a feature quantity. Thecomputer 100-2 includes a contribution value calculation unit 120 and acomparison target data selection unit 121, and stores comparison targetdata management information 122.

The comparison target data management information 122 is information formanaging comparison target data 400 (see FIG. 4). Here, the comparisontarget data 400 is data serving as a reference for characterizing theevaluation target data 300 when calculating a contribution value of eachfeature quantity of the evaluation target data 300. The learning datamay be set as the comparison target data 400, or the comparison targetdata 400 that is artificially generated using a mode value and anaverage value of the learning data may be set.

When the contribution value of the feature quantity contained in theevaluation target data 300 is calculated without using the comparisontarget data 400, the computer 100-2 may not store the comparison targetdata management information 122.

The contribution value calculation unit 120 calculates, as thecontribution value, a value indicating how much the evaluation targetdata 300 contributes to the predicted value by having such a featurequantity as compared with the comparison target data 400.

The comparison target data selection unit 121 selects, from thecomparison target data management information 122, the comparison targetdata 400 to be used for calculating a contribution value of a featurequantity contained in the evaluation target data 300.

The computer 100-3 outputs information for interpreting a basis ofoutput of a predicted value of the evaluation target data 300. Thecomputer 100-3 includes an interpretation factor selection unit 130 anda result output unit 131 and stores interpretation factor conversioninformation 132.

The interpretation factor conversion information 132 is information formanaging an interpretation factor which is determined by a value and acontribution value of a feature quantity contained in the evaluationtarget data 300. As to be described below, the computer 100-3 outputs,as information to be presented to the user, information including theinterpretation factor and a score. Here, the interpretation factor is aspecific factor in accordance with a target or an event of a target. Inthe present embodiment, natural language information that can beunderstood by the user is set as an interpretation factor. The score isa value for evaluating validity of the interpretation factor.

The interpretation factor selection unit 130 specifies a correspondinginterpretation factor by referring to the interpretation factorconversion information 132 based on the value and the contribution valueof the feature quantity, and calculates a score of the interpretationfactor.

The result output unit 131 generates display information for presentinginformation on the predicted value and the interpretation factor of theevaluation target data 300 to the user, and transmits the displayinformation to the terminal 101.

It should be noted that any one of the computers 100-1, 100-2, and 100-3has an operation receiving unit that provides an application programminginterface (API) for receiving a request, data, and the like from theterminal 101.

Here, a hardware configuration of the computer 100 will be described.FIG. 2 shows an example of the hardware configuration of the computer100 according to the first embodiment.

The computer 100 includes a processor 201, a main storage device 202, asecondary storage device 203, a network interface 204, an input device205, and an output device 206. The respective hardware components areconnected to one another via an internal bus. The computer 100 may notinclude any one of the secondary storage device 203, the input device205, and the output device 206.

The processor 201 executes a program stored in the main storage device202. The processor 201 operates as a functional unit (module) thatimplements a specific function, such as the contribution valuecalculation unit 120, by executing processing in accordance with theprogram. In the following description, when processing is described interms of a functional unit, it is indicated that the processor 201executes a program that implements the functional unit.

The main storage device 202 stores the program to be executed by theprocessor 201 and information to be used by the program. The mainstorage device 202 includes a work area to be temporarily used by theprogram.

Program and model information for implementing the predictor 110 isstored in the main storage device 202 of the computer 100-1. A programfor implementing the contribution value calculation unit 120 and thecomparison target data selection unit 121 is stored in the main storagedevice 202 of the computer 100-2. A program for implementing theinterpretation factor selection unit 130 and the result output unit 131is stored in the main storage device 202 of the computer 100-3. The mainstorage device 202 of any one of the computers 100-1, 100-2, and 100-3stores a program for implementing the operation receiving unit.

The secondary storage device 203, such as a hard disk drive (HDD) and asolid state drive (SSD), stores data permanently.

The secondary storage device 203 of the computer 100-2 stores thecomparison target data management information 122. The secondary storagedevice 203 of the computer 100-3 stores the interpretation factorconversion information 132.

For the functional units provided in each of the computers 100, theplurality of functional units may be integrated into one functionalunit, or one functional unit may be divided into a plurality offunctional units for each function. Each of the functional units may beintegrated into one computer 100.

FIG. 3 shows an example of a data structure of the evaluation targetdata and the output data according to the first embodiment.

The predictor 110 processes the evaluation target data 300 including anID 301 and feature quantities 302 based on the model information, andoutputs output data 310 including an ID 311 and an output value 312 as aprediction result.

In the example shown in FIG. 3, output data 310-1 is output fromevaluation target data 300-1, output data 310-2 is output from theevaluation target data 300-2, and output data 310-M is output fromevaluation target data 300-M.

A field of the ID 301 stores identification information of theevaluation target data 300. In the ID 301 according to the firstembodiment, a number is stored.

A field group of the feature quantity 302 stores values of featurequantities contained in the evaluation target data 300. The featurequantity is, for example, gender, age, annual income, and depositamount. One of descriptions “male” and “female” is stored in a field ofthe feature quantity corresponding to the gender, and a numerical valueis stored in a field of the feature quantity corresponding to the age.

A field of the ID 311 stores identification information of the outputdata 310. It is assumed that the same value as that of ID 301 is set inthe ID 311. A field of the output value 312 stores a predicted valueoutput by the predictor 110.

FIG. 4 shows an example of a data structure of the comparison targetdata management information 122 according to the first embodiment.

The comparison target data management information 122 includes an entry400 including an ID 401 and feature quantities 402. One entry 400corresponds to one piece of comparison target data. In the followingdescription, the entry 400 is also referred to as comparison target data400.

A field of the ID 401 stores identification information of thecomparison target data 400. In the field of the ID 401 according to thefirst embodiment, a number is stored. The number stored in the field ofthe ID 401 is not related to the numbers stored in the field of the ID301 and the field of the ID 311.

A field group of the feature quantity 402 stores values of featurequantities contained in the comparison target data 400. The featurequantities contained in the field group of the feature quantity 402 arethe same as the feature quantities contained in the field group of thefeature quantity 302.

FIG. 5 shows an example of a data structure of the interpretation factorconversion information 132 according to the first embodiment.

The interpretation factor conversion information 132 is information inwhich a set including, as elements, a combination of a value andcontribution value of the feature quantity contained in the input datais associated with an interpretation factor. The interpretation factorconversion information 132 according to the first embodiment includes aconversion table 500 associated with identification information of therespective feature quantities.

The conversion table 500 denotes data in a row-column format in which anarea of a value of a feature quantity is defined as a row component andan area of a contribution value of a feature quantity is defined as acolumn component. The conversion table 500 denotes information forspecifying an interpretation factor by using a set, having onecombination as an element, as a retrieval key.

A piece of interpretation factor data 510 is stored in at least one cellof the conversion table 500. It should be noted that two or more piecesof interpretation factor data 510 may be stored in one cell. Theinterpretation factor data 510 includes an interpretation factor 511 anda weight 512.

The interpretation factor 511 is natural language information. Theinterpretation factor data 510 including the same interpretation factors511 is set in the plurality of conversion tables 500, whereby enablinginterpretation of an abstraction about what the evaluation target data300 determined by the value and the contribution value of each featurequantity generally means.

The weight 512 is a value representing the degree of validitycorresponding to the interpretation factor. As will be described below,a score of the interpretation factor is calculated using the weight 512.It should be noted that the weight (score) may not be taken intoconsideration when all the weights 512 of all the conversion tables 500are set to be the same.

The interpretation factor conversion information 132 may be informationthat can specify an interpretation factor based on a value and acontribution value of a feature quantity. For example, instead of theconversion table 500, data in a format of a table which stores data inwhich a set including a plurality of combinations as elements isassociated with an interpretation factor, may be used.

It is assumed that the interpretation factor conversion information 132is set in advance in the first embodiment. A method of generating theinterpretation factor conversion information 132 will be described in asecond embodiment.

FIG. 6 shows a flow of processing of the computer system according tothe first embodiment. Arrows in the drawing indicate the flow of data.FIG. 7 shows an example of a data structure of contribution value data600 according to the first embodiment. FIG. 8 shows an example of a datastructure of interpretation information 610 according to the firstembodiment. FIGS. 9A and 9B shows an example of an input screen 900displayed on the terminal 101 according to the first embodiment.

First, the data structures of the data and the information handled inthe processing executed by the computer system will be described.

The contribution value data 600 includes an ID 701 and contributionvalues 702.

A field of the ID 701 stores identification information of thecontribution value data 600. In the field of the ID 701, the same valueas in the field of the ID 301 of the evaluation target data 300 is set.

A field group of the contribution value 702 stores the respectivecontribution values of the plurality of feature quantities contained inthe evaluation target data 300. As will be described below, acontribution value is calculated using a known calculation method. Thepresent invention is not limited to the calculation method of acontribution value.

The interpretation information 610 includes an entry including aninterpretation factor 801 and a score 802. One entry corresponds to oneinterpretation factor.

A field of the interpretation factor 801 stores the interpretationfactor specified based on the interpretation factor conversioninformation 132. A field of the score 802 stores a score.

Next, a flow of processing of the computer system will be described.First, the operation receiving unit presents the input screen 900 to theterminal 101. Here, the input screen 900 will be described.

The input screen 900 is a screen presented by the operation receivingunit, and is displayed on the terminal 101. The input screen 900includes an evaluation target data setting field 901, a comparisontarget data setting field 902, an interpretation factor informationsetting field 903, a confirmation button 904, and an execute button 905.

The evaluation target data setting field 901 is a field for designatingthe evaluation target data 300. The comparison target data setting field902 is a field for designating the comparison target data 400. Theinterpretation factor information setting field 903 is a field fordesignating the interpretation factor conversion information 132.

The confirmation button 904 is an operation button for confirming theconversion table 500 stored in the interpretation factor conversioninformation 132 specified in the interpretation factor informationsetting field 903. When the confirmation button 904 is operated, aconversion table confirmation field 906 is displayed on the input screen900. The conversion table confirmation field 906 includes a featurequantity selection field 907.

The feature quantity selection field 907 is a field for selecting afeature quantity. A conversion table 500 corresponding to the featurequantity set in the feature quantity selection field 907 is displayed inthe conversion table confirmation field 906.

Only a cell in which the interpretation factor data 510 is stored may bedisplayed in the conversion table confirmation field 906. It should benoted that all the conversion tables 500 may be displayed in theconversion table confirmation field 906. In this case, the conversiontable confirmation field 906 does not include the feature quantityselection field 907.

The execution button 905 is a button for instructing generation of theinterpretation information 610 of the evaluation target data 300. Whenthe execution button 905 is operated, the terminal 101 transmits aninterpretation information output request including information on theevaluation target data 300, the comparison target data 400, and theinterpretation factor conversion information 132, which are designatedby the user.

When the comparison target data 400 is generated from statistics such asthe average value and the mode value of the evaluation target data 300that was input in the past, or is randomly generated, the input screen900 may not include the comparison target data setting field 902.

The input screen 900 has been described above. The description will nowreturn to FIG. 6.

When receiving the interpretation information output request, theoperation receiving unit outputs a calculation instruction of apredicted value to the computer 100-1, and outputs a calculationinstruction of a contribution value of the evaluation target data 300 tothe computer 100-2. The calculation instruction of a predicted valueincludes designation information of the evaluation target data 300, andthe calculation instruction of a contribution value includes designationinformation of the comparison target data 400. The operation receivingunit outputs a generation instruction of the interpretation information610 to the computer 100-3. The generation instruction of theinterpretation information 610 includes designation information of theevaluation target data 300.

The predictor 110 of the computer 100-1 generates the output data 310 byprocessing the evaluation target data 300 based on the modelinformation. The predictor 110 outputs the output data 310 to thecomputer 100-3.

The contribution value calculation unit 120 of the computer 100-2transmits a selection instruction of the comparison target data 400 tothe comparison target data selection unit 121. The selection instructionof the comparison target data 400 includes designation information ofthe comparison target data 400. When receiving the selection instructionof the comparison target data 400, the comparison target data selectionunit 121 of the computer 100-2 acquires the comparison target data 400to be used from the comparison target data management information 122based on the designation information of the comparison target data 400.The comparison target data selection unit 121 outputs the comparisontarget data 400 acquired by the contribution value calculation unit 120.

When receiving the comparison target data 400, the contribution valuecalculation unit 120 calculates the contribution value of each featurequantity of the evaluation target data 300 based on the evaluationtarget data 300 and the comparison target data 400, and generates thecontribution value data 600. The contribution value calculation unit 120outputs the contribution value data 600 to the computer 100-3.

When receiving the output data 310, the contribution value data 600, andthe generation instruction of the interpretation information 610, theinterpretation factor selection unit 130 of the computer 100-3 generatesthe interpretation information 610, based on the evaluation target data300 and the contribution value data 600, by referring to the conversiontable 500 stored in the interpretation factor conversion information132. The interpretation factor selection unit 130 outputs the evaluationtarget data 300, the output data 310, the contribution value data 600,and the interpretation information 610 to the result output unit 131.

The result output unit 131 of the computer 100-3 generates displayinformation 620 for presenting the evaluation target data 300, theoutput data 310, the contribution value data 600, and the interpretationinformation 610 to the user. The result output unit 131 outputs thedisplay information 620 to the operation receiving unit. The operationreceiving unit transmits the display information 620 to the terminal101.

In FIG. 6, the predicted value and the interpretation information 610are presented in a series of processing. However, presentation of thepredicted value and presentation of interpretation information 610 maybe performed as separate processing. In this case, the input screen 900may be divided into a field including an evaluation target data settingfield 901, a comparison target data setting field 902, and an executionbutton for prediction, and a field including an interpretation factorinformation setting field 903, a confirmation button 904, and anexecution button for interpretation.

Next, the processing executed in the computer system will be describedin detail. First, the generation processing of the contribution valuedata 600 will be described.

In the first embodiment, the contribution value is calculated using acalculation method described in Non-Patent Literature 1. Here, a summaryof the calculation method of a contribution value will be described.FIG. 10 is a flowchart showing an example of contribution value datageneration processing executed by the contribution value calculationunit 120 according to the first embodiment. FIGS. 11A, 11B, 11C, 11D,and 11E show an example of a data structure of the intermediate data1100 according to the first embodiment.

The contribution value calculation unit 120 generates subsets of thefeature quantities (step S101). It should be noted that the subset ofthe feature quantities includes an empty set as well.

Next, the contribution value calculation unit 120 starts loop processingof the subsets of feature quantities (step S102).

Specifically, the contribution value calculation unit 120 selects onesubset from unprocessed subsets. The contribution value calculation unit120 selects an empty set as an initial value.

Next, the contribution value calculation unit 120 generates theintermediate data 1100 based on the evaluation target data 300, thecomparison target data 400, and a subset K, and outputs the generatedintermediate data 1100 to the predictor 110 (step S103). Thecontribution value calculation unit 120 shifts to awaiting state untilthe output data 310 is output from the predictor 110.

Specifically, the contribution value calculation unit 120 generates theintermediate data 1100 by setting a value of a feature quantity of theevaluation target data 300 in the feature quantity contained in thesubset, and setting a value of a feature quantity of the comparisontarget data 400 in the feature quantity not contained in the subset.

For example, when the evaluation target data 300-1 and the comparisontarget data 400 whose ID 401 is “1” is used, the intermediate data 1100as shown in FIGS. 11A, 11B, 11C, 11D, and 11E is generated.

When the empty set is selected, intermediate data 1100-1 shown in FIG.11A is generated. When a subset including the feature quantity “gender”as an element is selected, intermediate data 1100-2 shown in FIG. 11B isgenerated. When a subset including the feature quantity “age” as anelement is selected, intermediate data 1100-3 shown in FIG. 11C isgenerated. When a subset including the feature quantities “gender” and“age” is selected, intermediate data 1100-4 shown in FIG. 11D isgenerated. When a subset including all the feature quantities aselements is selected, intermediate data 1100-5 shown in FIG. 11E isgenerated.

When receiving the intermediate data 1100, the predictor 110 generatesthe output data 310 by processing the intermediate data 1100 based onthe model information. The predictor 110 outputs the output data 310 tothe contribution value calculation unit 120.

The contribution value calculation unit 120 acquires the output data 310output from the predictor 110 as intermediate output data (step S104),and stores the combination of the subset, the intermediate data 1100,and the intermediate output data in a work area (step S105).

Next, the contribution value calculation unit 120 determines whether theprocessing is completed for all subsets (step S106).

When it is determined that the processing is not completed for all thesubsets, the contribution value calculation unit 120 returns to stepS102 and selects a new subset.

When it is determined that the processing is completed for all thesubsets, the contribution value calculation unit 120 calculates thecontribution value of each feature quantity of the evaluation targetdata 300 by using the intermediate output data, and generates thecontribution value data 600 from the calculated contribution value (stepS107). Then, the contribution value calculation unit 120 terminates thecontribution value data generation processing. Specifically, thefollowing processing is executed.

(Processing A1) The contribution value calculation unit 120 selects atarget feature quantity from the feature quantities contained in theevaluation target data 300.

(Processing A2) The contribution value calculation unit 120 multiplies adifference between the intermediate output data corresponding to asubset including the target feature quantity as an element and theintermediate output data corresponding to a subset not including thetarget feature quantity as an element by any weights for a combinationof sets of feature quantities other than the target feature quantity,and adds up these values.

For example, a difference between the number of elements in the subsetand the number of all the feature quantities can be used as a weight. Avalue which is obtained by dividing a value, obtained by multiplying afactorial of a value obtained by subtracting one from the number ofelements in the subset by a factorial of the difference between thenumber of all the feature quantities and the number of elements in thesubset, by a factorial of the number of all the feature quantities, canbe used as a weight.

FIG. 12 is a flowchart showing an example of interpretation informationgeneration processing executed by the interpretation factor selectionunit 130 according to the first embodiment.

The interpretation factor selection unit 130 generates a list ofinterpretation factors set in the interpretation factor conversioninformation 132 by referring to the interpretation factor conversioninformation 132 (step S201).

Specifically, in the interpretation factor conversion information 132, alist including an entry including an interpretation factor and a scoreis generated. Initial values of scores of all the interpretation factorsare set as “0”.

Next, the interpretation factor selection unit 130 starts loopprocessing of a feature quantity (step S202).

Specifically, the interpretation factor selection unit 130 selects onefeature quantity from unprocessed feature quantities as a target featurequantity.

Next, the interpretation factor selection unit 130 acquires a conversiontable 500 of the target feature quantity from the interpretation factorconversion information 132 (step S203).

Next, the interpretation factor selection unit 130 specifies a cell ofthe conversion table 500 based on a value and a contribution value ofthe target feature quantity contained in the evaluation target data 300(step S204).

Specifically, the interpretation factor selection unit 130 specifies acolumn corresponding to an area including the value of the targetfeature quantity, and specifies a row corresponding to an area includingthe contribution value of the target feature quantity.

Next, the interpretation factor selection unit 130 determines whetherthe interpretation factor data 510 exists in the specified cell (stepS205).

When it is determined that the interpretation factor data 510 does notexist in the specified cell, the interpretation factor selection unit130 proceeds to step S207.

When it is determined that the interpretation factor data 510 exists inthe specified cell, the interpretation factor selection unit 130 updatesa score of an interpretation factor corresponding to the interpretationfactor data 510 (step S206). Thereafter, the interpretation factorselection unit 130 proceeds to step S207.

Specifically, the interpretation factor selection unit 130 retrieves anentry corresponding to the interpretation factor 511 of theinterpretation factor data 510 by referring to the list, and adds avalue of the weight 512 to a score of the retrieved entry.

In step S207, the interpretation factor selection unit 130 determineswhether the processing is completed for all the feature quantities (stepS207).

When it is determined that the processing is not completed for all thefeature quantities, the interpretation factor selection unit 130 returnsto step S202 and selects a new feature quantity.

When it is determined that the processing is completed for all thefeature quantities, the interpretation factor selection unit 130generates the interpretation information 610 based on the list (stepS208). Then, the interpretation factor selection unit 130 terminates theinterpretation information generation processing.

For example, the interpretation factor selection unit 130 directlygenerates the list as the interpretation information 610. Theinterpretation factor selection unit 130 may select only an entry whosescore is larger than a threshold, and generate the interpretationinformation 610 from the selected entry. The interpretation factorselection unit 130 may select any number of entries in descending orderof the score, and generate the interpretation information 610 from theselected entry.

FIG. 13 shows an example of a result output screen 1300 displayed on theterminal 101 according to the first embodiment.

The operation receiving unit presents the result output screen 1300 onthe terminal 101 based on the display information 620. The result outputscreen 1300 includes a predicted value display field 1301 and aninterpretation factor display field 1302.

The predicted value display field 1301 is a field for displaying apredicted value of the evaluation target data 300. The interpretationfactor display field 1302 is a field for displaying an interpretationfactor. In the interpretation factor display field 1302, the score 802corresponding to the interpretation factor 801 of each entry of theinterpretation information 610 is displayed as a bar graph.

The result output unit 131 may generate display information 620 fordisplaying, in the interpretation factor display field 1302, an entrywhose value of the score 802 is larger than a threshold. The resultoutput unit 131 may generate display information 620 for selecting anynumber of entries in descending order of the score 802 and displayingthe selected entries in the interpretation factor display field 1302.

As described above, according to the first embodiment, the computersystem can present the interpretation factor to the user as informationuseful for understanding the basis of the predicted value, together withthe predicted value (prediction result) of the evaluation target data300. The computer system can also present a score along with aninterpretation factor.

The user can interpret the prediction basis with a certain degree ofsatisfaction by interpreting the abstraction about what characteristicsand meaning the evaluation target data 300 has. In addition, the usercan quantitatively evaluate the validity of each interpretation factor.

Second Embodiment

In the second embodiment, a method of generating the conversion table500 will be described. Hereinafter, the second embodiment will bedescribed focusing on differences from the first embodiment.

FIG. 14 shows an example of a configuration of a computer systemaccording to the second embodiment.

The configuration of the computer system according to the secondembodiment is the same as that of the first embodiment. The hardwareconfiguration of the computer 100 according to the second embodiment isthe same as that of the first embodiment. The software configurations ofthe computers 100-1 and 100-3 according to the second embodiment are thesame as those of the first embodiment. In the second embodiment, thesoftware configuration of the computer 100-2 is different.

The computer 100-2 newly includes a conversion information generationunit 1400, and further stores intermediate calculation data managementinformation 1410. The conversion information generation unit 1400includes a candidate interpretation factor generation unit 1401 and aconversion table generation unit 1402.

FIG. 15 shows a flow of processing of the computer system according tothe second embodiment. Arrows in the figure indicate the flow of data.FIG. 16 shows an example of a data structure of the data for generation1500 according to the second embodiment. FIG. 17 shows an example of adata structure of intermediate calculation data 1510 according to thesecond embodiment. FIG. 18 shows an example of a data structure of areadivision information 1520 according to the second embodiment. FIG. 19shows an example of a data structure of candidate interpretation factorinformation 1530 according to the second embodiment. FIG. 20 shows anexample of a data structure of a candidate conversion table 1540according to the second embodiment. FIG. 21 shows an example of asetting screen 2100 displayed on the terminal 101 according to thesecond embodiment. FIG. 22 shows an example of a division conditiondetail setting screen 2200 displayed on the terminal 101 according tothe second embodiment. FIG. 23 shows an example of a confirmation screen2300 displayed on the terminal 101 according to the second embodiment.

First, the data structures of the data and the information handled inthe processing executed by the computer system will be described.

The data for generation 1500 has the same data structure as theevaluation target data 300, and includes an ID 1601 and featurequantities 1602. A field of the ID 1601 stores identificationinformation of the data for generation 1500. In the field of the ID 1601according to the second embodiment, a number is stored. A field of thefeature quantity 1602 is the same as the field of the feature quantity302.

The intermediate calculation data 1510 includes an ID 1701, a referenceID 1702, a subset 1703, feature quantities 1704, and an output value1705.

A field of the ID 1701 stores identification information of theintermediate calculation data 1510. It is assumed that the sameidentification information as the intermediate data 1100 is set in theID 1701. In the present embodiment, the intermediate data 1100 isgenerated based on the data for generation 1500 and the comparisontarget data 400.

A field of the reference ID 1702 stores identification information ofthe data for generation 1500 used at the time of generating theintermediate data 1100. The field of the reference ID 1702 stores thesame value as that of the ID 1601.

A field of the subset 1703 stores a combination of feature quantities ofthe data for generation 1500 whose values were set at the time ofgenerating the intermediate data 1100. When the field of the subset 1703is empty, the intermediate data 1100 matches the comparison target data400. On the other hand, when all the feature quantities are set in thesubset 1703, the intermediate data 1100 matches the data for generation1500.

A field group of the feature quantity 1704 stores values of featurequantities constituting the intermediate data 1100. A field of theoutput value 1705 stores a value output by the predictor 110 to whichthe intermediate data 1100 is input.

The area division information 1520 stores entries each including afeature quantity 1801, a division area (value) 1802, and a division area(contribution value) 1803. One entry corresponds to one featurequantity.

A field of the feature quantity 1801 stores identification informationof a feature quantity. The feature quantity 1801 is also used asidentification information of an entry.

A field of the division area (value) 1802 stores information on an areagenerated by dividing a value range of the value of the featurequantity. In the field of division area (value) 1802, data in whichidentification information of the area is associated with a range of thevalue or a type of the value is stored as many as the number of areas.

A field of the division area (contribution value) 1803 storesinformation on an area generated by dividing a value range of thecontribution value of the feature quantity. In the field of the divisionarea (contribution value) 1803, data in which identification informationof the area is associated with a range of the contribution value isstored as many as the number of areas.

The candidate interpretation factor information 1530 stores entries eachincluding an ID 1901, a feature quantity 1902, an area (value) 1903, anarea (contribution value) 1904, a weight 1905, the number of pieces offirst data 1906, the number of pieces of second data 1907, aninteraction value 1908, and an interpretation factor 1909. One entrycorresponds to one candidate interpretation factor.

A field of the ID 1901 stores identification information of a candidateinterpretation factor. The field of the ID 1901 corresponds to aninterpretation factor ID 2001.

A field of the feature quantity 1902, a field of the area (value) 1903,and a field of the area (contribution value) 1904 stores informationindicating classification criteria of the candidate interpretationfactors.

The field of the feature quantity 1902 stores identification informationof a feature quantity serving as a classification criterion of thecandidate interpretation factors. One or more feature quantities are setin one entry.

The field of the area (value) 1903 stores an area of a value of afeature quantity corresponding to the feature quantity 1902. The fieldof the area (contribution value) 1904 stores an area of a contributionvalue of a feature quantity corresponding to the feature quantity 1902.

The data for generation 1500, in which a value of a feature quantitycorresponding to the feature quantity 1902 is contained in the area(value) 1903, and a contribution value of a feature quantitycorresponding to the feature quantity 1902 is contained in the area(contribution value) 1904, is classified as data corresponding to thecandidate interpretation factor corresponding to the entry.

A field of the weight 1905 stores a weight that is a value representingthe degree of the validity corresponding to the candidate interpretationfactor. When the weight is not taken into consideration, the field ofthe weight 1905 may not be contained.

A field of the number of pieces of the first data 1906 stores the numberof pieces of data for generation 1500 satisfying the classificationcriteria set in the entry, that is, the number of pieces of data forgeneration 1500 corresponding to the candidate interpretation factorcorresponding to the entry. The field of the number of pieces of thefirst data 1906 may store a ratio of the number of pieces of the datafor generation 1500 corresponding to the candidate interpretation factorrelative to the total number of pieces of the data for generation 1500.

A field of the number of pieces of the second data 1907 stores thenumber of pieces of data for generation 1500 corresponding only to thecandidate interpretation factor corresponding to the entry. The field ofthe number of pieces of the second data 1907 may store a ratio of thenumber of pieces of the data for generation 1500 corresponding only tothe candidate interpretation factor relative to the total number ofpieces of the data for generation 1500.

A field of the interaction value 1908 stores an interaction value of aset of the feature quantities 1902 in the data for generation 1500corresponding to the candidate interpretation factor. In the field ofthe interaction value 1908, a combination of the identificationinformation of the data for generation 1500 and the interaction value isstored as many as the number of pieces of the first data.

Here, the interaction value (second evaluation value) is a value thatrepresents magnitude of contribution of the set of values of featurequantities and that corresponds to the output value of the data forgeneration 1500. The interaction values can be calculated by usingmethods described in Lundberg, Scott M., Gabriel G. Erion, and Su-InLee. “Consistent Individualized Feature Attribution for Tree Ensembles.”ArXiv preprint arXiv: 1802.03888 (2018) (Non-Patent Literature 2) andGrabisch, Michel. “K-order additive discrete fuzzy measures and theirrepresentation.” Fuzzy sets and systems 92.2 (1997): 167-189.(Non-Patent Literature 3). A value, which indicates what kind of featurequantity the data for generation 1500 has as compared with thecomparison target data 400 and how much the fact contributes to theoutput value, is calculated as an interaction value.

A field of the interpretation factor 1909 stores an interpretationfactor (natural language information) set for the candidateinterpretation factor. At a time point when the candidate interpretationfactor information 1530 is generated, the field of the interpretationfactor 1909 of each entry is blank. The setting of the value to thefield of the interpretation factor 1909 is performed by the user.

The candidate conversion table 1540 has the same data format as theconversion table 500. Candidate interpretation factor data 2000 isstored in at least one cell of the candidate conversion table 1540. Twoor more pieces of candidate interpretation factor data 2000 may bestored in a cell.

The candidate interpretation factor data 2000 includes an interpretationfactor ID 2001 and a weight 2002. The interpretation factor ID 2001 isidentification information of the candidate interpretation factor. Theweight 2002 is the same as the weight 512.

Next, a flow of processing of the computer system will be described.First, the operation receiving unit presents the setting screen 2100 tothe terminal 101. Here, the setting screen 2100 and the divisioncondition detail setting screen 2200 will be described.

The setting screen 2100 is a screen presented by the operation receivingunit, and is displayed on the terminal 101. The setting screen 2100includes a data set setting field 2101, a generation condition settingfield 2102, an adoption condition setting field 2103, a divisioncondition setting field 2104, and an execution button 2105.

The data set setting field 2101 is a field for designating data to beused for processing. A value set in the data set setting field 2101 isoutput as data set information. The data set setting field 2101 includesa data for generation setting field 2111 and a comparison target datasetting field 2112.

The data for generation setting field 2111 is a field for designatingthe data for generation 1500 to be used. The comparison target datasetting field 2112 is a field for designating the comparison target data400.

When the comparison target data 400 is generated from statistics such asthe average value and the mode value of the evaluation target data 300that was input in the past, or is randomly generated, the comparisontarget data setting field 2112 may not be contained.

The generation condition setting field 2102 is a field for setting ageneration condition that defines constraints on generation of candidateinterpretation factors. The values set in the generation conditionsetting field 2102 are output as generation condition information. Thegeneration condition setting field 2102 includes a radio button 2121 andcheck fields 2122, 2123, 2124, 2125, and 2126.

The radio button 2121 is a button that is selected when the generationcondition is not set. In this case, inputs to the check fields 2122,2123, 2124, 2125, and 2126 are invalidated.

The check field 2122 is a field to be operated to set an upper limitvalue of the number of the candidate interpretation factors. The useroperates the check field 2122 to set a value equal to or larger than 1.

The check field 2123 is a field to be operated to set an upper limitvalue of calculation time of the processing of generating the conversiontable 500. The user operates the check field 2123 to set a value equalto or larger than 0. The upper limit value of the calculation time maybe set for specific processing.

The check field 2124 is a field to be operated to set an upper limitvalue of the number of feature quantities to be contained as aclassification criterion for one interpretation factor. The useroperates the check field 2124 to set a value equal to or larger than 1.

The check field 2125 is a field to be operated to set a lower limitvalue of the number of feature quantities to be contained as aclassification criterion for one interpretation factor. The useroperates the check field 2125 to set a value equal to or larger than 1.

The check field 2126 is a field to be operated to set a lower limitvalue of the number of the data for generation 1500 corresponding to thecandidate interpretation factor. The user operates the check field 2126to set a value equal to or larger than 1.

The adoption condition setting field 2103 is a field for setting anadoption condition of the interpretation factor based on an interactionvalue. A value set in the adoption condition setting field 2103 isoutput as adoption condition information. The adoption condition settingfield 2103 includes radio buttons 2131, 2132 and 2133.

The radio button 2131 is a radio button operated to use a presetadoption condition. In the present embodiment, the candidateinterpretation factors are determined in descending order of theinteraction values within a range that satisfies the generationcondition.

The radio button 2132 is a radio button operated to use the adoptioncondition based on a ratio between the interaction value and thepredicted value. The user operates the radio button 2132 to set a valueequal to or larger than 0.

The radio button 2133 is a radio button operated to use the adoptioncondition based on a significance level. The user operates the radiobutton 2133 to set a value larger than 0 and smaller than 1. A valueused in other hypothesis test methods, such as a p value, may be set inthe radio button 2133.

The division condition setting field 2104 is a field for settingdivision conditions for the value ranges of the value and thecontribution value of the feature quantity. The value set in thedivision condition setting field 2104 is output as the divisioncondition information. The division condition setting field 2104includes a value division condition field 2140, a contribution valuedivision condition field 2150, and a detail setting button 2160.

The value division condition field 2140 is a field for setting thedivision condition of the value range of the value of the featurequantity, and includes radio buttons 2141, 2142, and 2143.

The radio button 2141 is a radio button operated to equally divide thevalue range of the value. The user operates the radio button 2141 to seta value equal to or larger than 1 as the number of divisions.

The radio button 2142 is a radio button operated to equally divide thevalue range of the value into two parts based on statistics such as amedian value, an average value, and a mode value. The user operates theradio button 2142 to set a reference statistic. It should be noted thatdifferent values may be set for the feature quantity depending on thedifference of a quantitative variable or a categorical variable.

The radio button 2143 is a radio button operated to equally divide thevalue range of the value based on the number of pieces of the data forgeneration 1500 contained in the area. The user operates the radiobutton 2143 to set a value equal to or larger than 1 as the number ofdivisions.

The contribution value division condition field 2150 is a field forsetting the division condition of the value range of the contributionvalue of the feature quantity, and includes radio buttons 2151, 2152,and 2153. The radio buttons 2151, 2152, and 2153 are similar to theradio buttons 2141, 2142, and 2143.

The detail setting button 2160 is a button operated to set a divisioncondition for each feature quantity. When the user operates the detailsetting button 2160, the division condition detail setting screen 2200is displayed. Details of the division condition detail setting screen2200 will be described below.

When the division condition information is preset, the divisioncondition setting field 2104 may not be contained in the setting screen2100. The value division condition field 2140 may be provided with radiobuttons for automatically setting a division condition corresponding toanyone of the radio buttons 2141, 2142, and 2143.

The execution button 2105 is a button for instructing generation of theconversion table 500. When the execution button 2105 is operated, theterminal 101 transmits a generation request of the conversion table 500that includes data set information, generation condition information,adoption condition information, and division condition information.

The setting screen 2100 may include an operation button for registering,in the computer 100-2, a value of each field as initial conditioninformation. The setting screen 2100 may include an operation button forinitializing the value of each field.

The setting screen 2100 has been described above. Next, the divisioncondition detail setting screen 2200 will be described.

The division condition detail setting screen 2200 includes a featurequantity confirmation field 2201, a division condition setting field2202, a return button 2203, a quantitative variable reflection button2204, and a reflection button 2205.

The feature quantity confirmation field 2201 is a field for displayinginformation on the feature quantity of the data for generation 1500. Thefeature quantity confirmation field 2201 includes a display field 2210and a selection button 2211.

The display field 2210 is a field for displaying information ondistribution of the data for generation 1500 related to a certainfeature quantity. A distribution graph 2220 is displayed in the displayfield 2210 according to the present embodiment.

The distribution graph 2220 is a graph such as a scatter graph or a heatmap which shows the distribution of the data for generation 1500 withthe value and the contribution value of the feature quantity as axes.

A value histogram 2221 is a histogram showing the number of pieces ofthe data for generation 1500 classified into categories of the value ofthe feature quantity. The categories are set at any width. Acontribution value histogram 2222 is a histogram showing the number ofpieces of the data for generation 1500 belonging to the categories ofthe contribution value of the feature quantity. The categories are setat any width.

The user can set the division conditions of the value ranges of thevalue and the contribution value of the feature quantity by referring tothe information in the display field 2210.

The selection button 2211 is an operation button for changing thefeature quantity to be displayed in the display field 2210. When aselection button 2211-1 is operated, information on a previous featurequantity of a current feature quantity is displayed in the display field2210, and when a selection button 2211-2 is operated, information on anext feature quantity of the current feature quantity is displayed inthe display field 2210.

The feature quantity confirmation field 2201 may include a field fordirectly designating a feature quantity instead of the selection button2211. The feature quantity confirmation field 2201 may include a fieldfor selecting feature quantities having a certain relation based on thevalue of the feature quantity, instead of the selection button 2211.When information of all the feature quantities can be displayed in thefeature quantity confirmation field 2201, the selection button 2211 maynot be contained in the feature quantity confirmation field 2201.

The division condition setting field 2202 is a field for settingdivision conditions of the value ranges of the value and thecontribution value of the feature quantity. The division conditionsetting field 2202 includes a value division condition field 2230 and acontribution value division condition field 2240.

The value division condition column 2230 is a field for setting thedivision condition of the value range of the value of the featurequantity, and includes radio buttons 2231, 2232, and 2233. The radiobuttons 2231, 2232, and 2233 are the same as the radio buttons 2141,2142, and 2143.

The contribution value division condition field 2240 is a field forsetting the division condition of the value range of the contributionvalue of the feature quantity, and includes radio buttons 2241, 2242,and 2243. The radio buttons 2241, 2242, and 2243 are the same as theradio buttons 2151, 2152, and 2153.

A return button 2203 is an operation button for switching a display fromthe division condition detail setting screen 2200 to the setting screen2100. When the return button 2203 is operated, the value set using thedivision condition detail setting screen 2200 is output as the divisioncondition information.

The quantitative variable reflect button 2204 is an operation button forsetting the value set in the division condition setting field 2202 as adivision condition of all the feature quantities corresponding to thequantitative variable. The division condition detail setting screen 2200may include a categorical variable reflect button instead of thequantitative variable reflect button 2204.

The reflect button 2205 is an operation button for setting the value setin the division condition setting field 2202 as the division conditionof all the feature quantities.

The division condition detail setting screen 2200 may include anoperation button for storing, in the computer 100-2, the divisioncondition information set via the division condition detail settingscreen 2200, and an operation button for instructing discarding orinitialization of the division condition information. The descriptionwill be returned to FIG. 15.

When receiving the generation request of the conversion table 500, theoperation reception unit outputs a calculation instruction of thepredicted value to the computer 100-1, and outputs the generationrequest of the conversion table 500 to the computer 100-2. Thecalculation instruction of the predicted value includes designationinformation of the data for generation 1500.

The predictor 110 of the computer 100-1 generates the output data 310 byprocessing the data for generation 1500 based on the model information.The predictor 110 outputs the output data 310 to the computer 100-2.

When receiving the generation request of the conversion table 500, thecontribution value calculation unit 120 of the computer 100-2 transmitsa selection instruction of the comparison target data 400 to thecomparison target data selection unit 121. The selection instruction ofthe comparison target data 400 includes designation information of thecomparison target data 400. When receiving the selection instruction ofthe comparison target data 400, the comparison target data selectionunit 121 acquires the comparison target data 400 to be used from thecomparison target data management information 122, based on thedesignation information of the comparison target data 400. Thecomparison target data selection unit 121 outputs the comparison targetdata 400 acquired by the contribution value calculation unit 120.

When receiving the comparison target data 400, the contribution valuecalculation unit 120 generates the contribution value data 600 based onthe data for generation 1500 and the comparison target data 400. Onepiece of contribution value data 600 is generated for one piece of datafor generation 1500. A method of generating the contribution value data600 is the same as that of the first embodiment, and a descriptionthereof will be omitted.

In addition, the contribution value calculation unit 120 stores, in theintermediate calculation data management information 1410, theintermediate data 1100 generated when the contribution value data 600for the data for generation 1500 is calculated, identificationinformation of the data for generation 1500, and intermediatecalculation data 1510 including the subset and the intermediate outputdata.

When receiving the output data 310 of the data for generation 1500, thecontribution value calculation unit 120 outputs a generation instructionof the conversion table 500 to the conversion information generationunit 1400. The generation instruction includes output data 310, thecontribution value data 600, the generation condition information, theadoption condition information, and the division condition information.

The candidate interpretation factor generation unit 1401 of theconversion information generation unit 1400 generates the area divisioninformation 1520 and the candidate interpretation factor information1530 based on the output data 310, the contribution value data 600, theintermediate calculation data 1510, the generation conditioninformation, the adoption condition information, and the divisioncondition information. The candidate interpretation factor generationunit 1401 outputs a selection instruction to the conversion tablegeneration unit 1402.

When the conversion table generation unit 1402 receives the selectioninstruction, the conversion table generation unit 1402 of the conversioninformation generation unit 1400 generates the candidate conversiontable 1540 based on the area division information 1520 and the candidateinterpretation factor information 1530, generates display informationfor presenting the candidate conversion table 1540 and the like, andoutputs the display information to the operation receiving unit. Theoperation receiving unit transmits the display information to theterminal 101.

The terminal 101 displays the confirmation screen 2300 based on thedisplay information generated by the conversion table generation unit1402. Here, the confirmation screen 2300 will be described.

The confirmation screen 2300 is a screen presented by the operationreceiving unit, and is displayed on the terminal 101. The confirmationscreen 2300 includes an interpretation factor information display field2301, a conversion table display field 2302, a candidate interpretationfactor display field 2303, and an Output Button 2304.

The interpretation factor information display field 2301 is a field fordisplaying a summary of the candidate interpretation factor, andincludes a data for generation display field 2311, a candidateinterpretation factor number field 2312, and an unclassified data numberfield 2313.

The data for generation display field 2311 displays a file or the likefor storing the data for generation 1500 to be used. The candidateinterpretation factor number field 2312 displays the number of generatedcandidate interpretation factors. The unclassified data number field2313 displays the number of pieces of data for generation 1500 notclassified as any of the generated candidate interpretation factors.

The conversion table display field 2302 displays a candidate conversiontable 1540, and includes a feature quantity selection field 2321 and acandidate conversion table display field 2322.

The feature quantity selection field 2321 is a field for selecting afeature quantity. The candidate conversion table display field 2322displays the candidate conversion table 1540 corresponding to thefeature quantity set in the feature quantity selection field 2321.

The conversion table display field 2302 may include an operation buttonsuch as a selection button 2211, instead of the feature quantityselection field 2321. When all the candidate conversion tables 1540 canbe displayed in the conversion table display field 2302, the featurequantity selection field 2321 may not be contained in the featurequantity confirmation field 2201.

The candidate interpretation factor display field 2303 displays detailedinformation of the candidate interpretation factor, and includes an IDselection field 2331, a first corresponding data number field 2332, asecond corresponding data number field 2333, a classification conditionconfirmation field 2334, an interpretation factor setting field 2335, asetting button 2336, and a discarding button 2337.

The ID selection field 2331 is a field for selecting a candidateinterpretation factor. As will be described below, a value correspondingto the ID 1901 is displayed in the candidate interpretation factor.

Information of the entries of the candidate interpretation factorinformation 1530 corresponding to the ID selection field 2331 isdisplayed in the first applicable data number field 2332, the secondapplicable data number field 2333, and the classification conditionconfirmation field 2334.

The first applicable data number field 2332 displays a value stored inthe first data number field 1906. The second applicable data numberfield 2333 displays a value stored in the second data number field 1907.

The classification condition confirmation field 2334 is a field fordisplaying information of a feature quantity that is a classificationcondition of the candidate interpretation factor, and includes featurequantity information 2341, a Delete Button 2342, and a Correct Button2343.

The feature quantity information 2341 is data in a format of a table inwhich entries each formed of four fields of a feature quantity, a valuearea, a contribution value area, and a weight are arranged in a columndirection. One column corresponds to a row of a feature quantitycontained in the entry of the candidate interpretation factorinformation 1530.

The field of the feature quantity includes a radio button. Any value canbe set by the user in the field of the weight.

The candidate interpretation factor display field 2303 may include anoperation button like the selection button 2211, instead of the IDselection field 2331. When information of all candidate interpretationfactors can be displayed in the candidate interpretation factor displayfield 2303, the ID selection field 2331 may not be contained in thecandidate interpretation factor display field 2303.

The Delete Button 2342 is an operation button for deleting an entry inwhich the radio button of the field of the feature quantity is operated.When the Delete Button 2342 is operated, the terminal 101 transmits adeletion request including the identification information of thecandidate interpretation factor and the identification information ofthe feature quantity which are designated by the user. In this case, theconversion table generation unit 1402 deletes the row of the featurequantity selected from the entries of the candidate interpretationfactor information 1530.

The Correct Button 2343 is an operation button for reflecting a userinput. When the Correct Button 2343 is operated, the terminal 101transmits a modification request including the identificationinformation of the candidate interpretation factor, the identificationinformation of the feature quantity, and the value of the weight, whichare designated by the user. In this case, the conversion tablegeneration unit 1402 updates the value of the weight 1905 of thecandidate interpretation factor information 1530 based on a correctcontent.

The interpretation factor setting field 2335 is a field for setting aninterpretation factor that is natural language information in thecandidate interpretation factor. The operation receiving unit maypresent, as auxiliary information for setting the interpretation factor,a generic name that affects the evaluation of the user of the output ofthe predictor 110 and the evaluation target data 300, using existingtechniques such as Internet Retrieval and Text Retrieval.

The setting button 2336 is an operation button for setting theinterpretation factor set in the interpretation factor setting field2335. When the setting button 2336 is operated, the terminal 101transmits a setting request including identification information of thecandidate interpretation factor and an interpretation factor, which aredesignated by the user. In this case, the conversion table generationunit 1402 sets the interpretation factor contained in the settingrequest to the field of the interpretation factor 1909 of the entry ofthe candidate interpretation factor information 1530.

The discarding button 2337 is an operation button for discarding thecandidate interpretation factor. When the discarding button 2337 isoperated, the terminal 101 transmits a discarding request includingidentification information of the candidate interpretation factordesignated by the user. In this case, the conversion table generationunit 1402 deletes the entry of the candidate interpretation factorinformation 1530. The conversion table generation unit 1402 updates thevalue of the number of candidate interpretations 2312 and the value ofthe number of pieces of unclassified data 2313 along with the deletionof the entry.

The output button 2304 is an operation button for instructing generationof the conversion table 500. When the output button 2304 is operated,the terminal 101 transmits a conversion table output request to theoperation receiving unit.

In the following description, a deletion request, a modificationrequest, a setting request, and a discarding request are notdistinguished and are also referred to as an information update request.The confirmation screen 2300 has been described above. The descriptionwill be returned to FIG. 15.

When the conversion table output request is received, the conversiontable generation unit 1402 generates the conversion table 500 based onthe area division information 1520 and the candidate interpretationfactor information 1530 updated based on the input of the user, andstores the conversion table 500 in the interpretation factor conversioninformation 132.

Next, the processing executed in the computer system will be describedin detail. First, processing executed by the candidate interpretationfactor generation unit 1401 will be described.

FIGS. 24A and 24B are flowcharts showing an example of processingexecuted by the candidate interpretation factor generation unit 1401according to the second embodiment. FIG. 25 is a diagram showing anexample of a data structure of the interaction value list 2500 accordingto the second embodiment.

The candidate interpretation factor generation unit 1401 generates thearea division information 1520 based on the area condition information,the data for generation 1500, and the contribution value data 600 (stepS301). Specifically, the following processing is executed.

(Processing B1) The candidate interpretation factor generation unit 1401selects a target feature quantity from the feature quantities containedin the data for generation 1500.

(Processing B2) The candidate interpretation factor generation unit 1401specifies, based on the data for generation 1500 and the contributionvalue data 600, the value range of a value of the target featurequantity and the value range of the contribution value.

(Processing B3) The candidate interpretation factor generation unit 1401divides the value range of the value of the target feature quantity intoa plurality of areas and divides the value range of the contributionvalue of the target feature quantity into a plurality of areas, based onthe division condition information. For example, the followingprocessing is executed.

When the radio button 2141 is operated, the candidate interpretationfactor generation unit 1401 equally divides the area of the value of thetarget feature quantity into the specified number. When the radio button2142 is operated, the candidate interpretation factor generation unit1401 calculates the statistic of the value of the target featurequantity, and equally divides the value area of the value into two partsbased on the statistic. When the radio button 2143 is operated, thecandidate interpretation factor generation unit 1401 divides the area ofthe value of the target feature quantity by the specified number so thatthe number of data for generation 1500 contained in each area is thesame or different.

(Processing B4) The candidate interpretation factor generation unit 1401adds an entry to the area division information 1520, and sets theidentification information of the target feature quantity in the featurequantity 1801 of the added entry. The candidate interpretation factorgeneration unit 1401 sets division results for the division area (value)1802 and the division area (contribution value) 1803 of the added entry.

(Processing B5) When the processing is not completed for all the featurequantities, the candidate interpretation factor generation unit 1401returns to (processing B1). When the processing is completed for all thefeature quantities, the candidate interpretation factor generation unit1401 terminates the processing in step S301.

The candidate interpretation factor generation unit 1401 may divide anarea based on a value such as the output value 1705 of the intermediatecalculation data 1510 as necessary. The processing in step S301 has beendescribed above.

Next, the candidate interpretation factor generation unit 1401 startsthe loop processing of the data for generation 1500 (step S302).

Specifically, the candidate interpretation factor generation unit 1401selects one data for generation 1500, from the unprocessed data forgeneration 1500, as the target data for generation 1500.

Next, the candidate interpretation factor generation unit 1401 startsthe loop processing of the number of feature quantities (step S303).

Specifically, the candidate interpretation factor generation unit 1401sets the value set in the check field 2125 to a variable n as an initialvalue. The candidate interpretation factor generation unit 1401initializes the interaction value list 2500. When the check field 2125is not operated, a preset value, for example, “2” is set therein.

The interaction value list 2500 includes entries formed of an ID 2501,an intermediate compute data ID 2502, a subset 2503, and an interactionvalue 2504. One entry corresponds to the interaction value data.

A field of the ID 2501 stores identification information of theinteraction value data. A field of the intermediate calculation data ID2502 stores identification information of the intermediate calculationdata 1510 used in calculating the interaction value. The field of theintermediate calculation data ID 2502 stores the same identificationinformation as that of the field of the ID 1701. The field of the subset2503 stores a subset of feature quantities as calculation target forinteraction values. A field of the interaction value 2504 stores theinteraction values.

In step S303, all entries in the interaction value list 2500 aredeleted. The processing in step S303 has been described above.

Next, the candidate interpretation factor generation unit 1401calculates an interaction value of a subset including n featurequantities (step S304). Specifically, the following processing isexecuted.

(Processing C1) The candidate interpretation factor generation unit 1401generates a subset including n feature quantities.

(Processing C2) The candidate interpretation factor generation unit 1401selects a target subset from the generated subsets.

(Processing C3) The candidate interpretation factor generation unit 1401calculates an interaction value of the target subset. The method ofcalculating the interaction value is described in Non-Patent Literature2 and Non-Patent Literature 3, so that a detailed description thereof isomitted. Alternatively, the interaction value is calculated as follows.

The candidate interpretation factor generation unit 1401 retrieves theintermediate calculation data 1510 in which the subset 1703 correspondsto the union of a subset M of complement of the target subset and asubset L of the target subset. The candidate interpretation factorgeneration unit 1401 calculates a weighted average of the output values1705 of the retrieved intermediate calculation data 1510 as aninteraction value.

The weight can be calculated as follows. (1) The factorial of a valueobtained by subtracting the total of the number of elements in thesubset M and the number of elements in the target subset from the totalnumber N of the feature quantities contained in the data for generationis multiplied by the factorial of the number of elements in the subset.(2) The value of (1) is divided by a value obtained by adding 1 to thevalue obtained by subtracting the number of elements in the targetsubset from N. (3) When the number of elements in a complement set ofthe subset L is an odd number, the value of (2) is multiplied by −1.

(Processing C4) The candidate interpretation factor generation unit 1401adds an entry to the interaction value list 2500, and sets theidentification information of the interaction value data in the field ofthe ID 2501 of the added entry. The candidate interpretation factorgeneration unit 1401 sets identification information of the intermediatecalculation data 1510 in the field of the intermediate calculation dataID 2502 of the added entry. The candidate interpretation factorgeneration unit 1401 sets a target subset in the field of the subset2503 of the added entry, and sets the calculated interaction value tothe field of the interaction value 2504.

(Processing C5) When the processing is not completed for all subsets,the candidate interpretation factor generation unit 1401 returns to(processing C2). When the processing is completed for all subsets, thecandidate interpretation factor generation unit 1401 terminates theprocessing in step S304. The processing in step S304 has been describedabove.

Next, the candidate interpretation factor generation unit 1401determines whether a subset satisfying the adoption condition exists(step S305). In the present embodiment, the determination in step S305is performed based on an interaction value. For example, the followingdetermination is performed in the present embodiment.

When the radio button 2131 is operated, the candidate interpretationfactor generation unit 1401 determines that the adoption condition issatisfied. In this case, a predetermined number of subsets having largevalues in the field of the interaction value 2504 are selected.

When the radio button 2132 is operated, the candidate interpretationfactor generation unit 1401 calculates a ratio of the interaction value2504 to the predicted value of the target data for generation 1500. Whenan entry whose ratio is larger than the threshold exists, the candidateinterpretation factor generation unit 1401 determines that a subsetsatisfying the adoption condition exists.

When the radio button 2133 is operated, the following determination isperformed. When the interaction value 2504 corresponding to a subset issignificantly larger than the interaction value 2504 corresponding toanother subset having the same number of elements as the subset with thespecific value of the interaction value 2504 being set as thesignificance level, the candidate interpretation factor generation unit1401 determines that a subset satisfying the adoption condition exists.

When it is determined that the subset satisfying the adoption conditiondoes not exist, the candidate interpretation factor generation unit 1401proceeds to step S309.

When it is determined that a subset satisfying the adoption conditionexists, the candidate interpretation factor generation unit 1401determines whether the candidate interpretation factor corresponding tothe subset is registered in the candidate interpretation factorinformation 1530 (step S306).

Specifically, the candidate interpretation factor generation unit 1401determines, by referring to the field of the area (value) 1903 and thefield of the area (contribution value) 1904, whether there exists anentry matching a combination of areas to which the values of the featurequantities constituting the subset satisfying the adoption conditionbelongs and a combination of areas to which the contribution values ofthe feature quantities constituting the subset satisfying the adoptioncondition belongs. When an entry satisfying the above condition exists,the candidate interpretation factor generation unit 1401 determines thatthe candidate interpretation factor corresponding to the subsetsatisfying the adoption condition is registered in the candidateinterpretation factor information 1530.

When it is determined that the candidate interpretation factorcorresponding to the subset satisfying the adoption condition isregistered in the candidate interpretation factor information 1530, thecandidate interpretation factor generation unit 1401 updates the entrycorresponding to the candidate interpretation factor (step S307), andthen the processing proceeds to step S309.

Specifically, the candidate interpretation factor generation unit 1401adds a combination of the identification information and the interactionvalue of the target data for generation 1500 selected in step S302 tothe field of the interaction value 1908 of the entry corresponding tothe candidate interpretation factor. The candidate interpretation factorgeneration unit 1401 adds 1 to the number of pieces of the first data1906 of the entry.

When it is determined that the candidate interpretation factorcorresponding to the subset satisfying the adoption condition is notregistered in the candidate interpretation factor information 1530, thecandidate interpretation factor generation unit 1401 adds an entrycorresponding to the candidate interpretation factor to the candidateinterpretation factor information 1530 (step S307), and then theprocessing proceeds to step S309. Specifically, the following processingis executed.

(Processing D1) The candidate interpretation factor generation unit 1401adds an entry to the candidate interpretation factor information 1530.The candidate interpretation factor generation unit 1401 sets theidentification information in the field of the ID 1901 of the addedentry, sets “1” in the field of the number of pieces of the first data1906, and sets “0” in the field of the number of pieces of the firstdata 1906. The candidate interpretation factor generation unit 1401 setsa combination of the identification information and the interactionvalue of the target data for generation 1500 selected in step S302 inthe field of the interaction value 1908 of the added entry.

(Processing D2) The candidate interpretation factor generation unit 1401generates the same number of rows as the number of elements of thetarget subset in the field of the feature quantity 1902, the field ofthe area (value) 1903, the field of the area (contribution value) 1904,and the field of the weight 1905 of the added entry. The candidateinterpretation factor generation unit 1401 sets the identificationinformation of the feature quantity, which is an element of the targetsubset, in the field of the feature quantity 1902 of the added entry.

(Processing D3) The candidate interpretation factor generation unit 1401selects a target feature quantity from the feature quantitiesconstituting the target subset. The candidate interpretation factorgeneration unit 1401 retrieves an entry whose feature quantity 1801corresponds to the target feature quantity, by referring to the areadivision information 1520.

(Processing D4) The candidate interpretation factor generation unit 1401specifies an area including the value of the target feature quantity ofthe target data for generation 1500 selected in step S302 by referringto the division area (value) 1802 of the retrieved entry. The candidateinterpretation factor generation unit 1401 sets the specified area inthe field of the area (value) 1903 of a row corresponding to the targetfeature quantity of the added entry.

(Processing D5) The candidate interpretation factor generation unit 1401specifies an area including a contribution value of the target featurequantity of the target data for generation 1500 selected in step S302 byreferring to the division area (contribution value) 1803 of theretrieved entry. The candidate interpretation factor generation unit1401 sets the specified area in the field of the area (contributionvalue) 1904 of a row corresponding to the target feature quantity of theadded entry.

(Processing D6) When the processing is not completed for all the featurequantities constituting the target subset, the candidate interpretationfactor generation unit 1401 returns to (processing D3). When theprocessing is completed for all the feature quantities constituting thetarget subset, the candidate interpretation factor generation unit 1401terminates the processing in step S307. The processing in step S307 hasbeen described above.

In step S309, the candidate interpretation factor generation unit 1401determines whether the termination condition is satisfied (step S309).For example, the following determination is performed in the presentembodiment.

When the check field 2122 is operated, the candidate interpretationfactor generation unit 1401 determines whether the number of entries ofthe candidate interpretation factor information 1530 (the number ofcandidate interpretation factors) matches the upper limit value. Whenthe number of entries of the candidate interpretation factor information1530 matches the upper limit value, the candidate interpretation factorgeneration unit 1401 determines that the termination condition issatisfied.

When the check field 2123 is operated, the candidate interpretationfactor generation unit 1401 determines whether the calculation time isequal to or longer than the threshold. When the calculation time isequal to or longer than the threshold, the candidate interpretationfactor generation unit 1401 determines that the termination condition issatisfied.

When it is determined that the end condition is satisfied, the candidateinterpretation factor generation unit 1401 stops the loop processing andthe processing proceeds to step S312. At this time, when the candidateinterpretation factor generated from the target data for generation 1500does not exist, the candidate interpretation factor generation unit 1401stores a combination of the identification information of the targetdata for generation 1500 and the unclassified flag in the work area.

When it is determined that the termination condition is not satisfied,the candidate interpretation factor generation unit 1401 determineswhether the number of feature quantities matches the upper limit value(step S310). When the check field 2124 is operated, the upper limitvalue is a value set in the check field 2124. When the check field 2124is not operated, the upper limit value is the total number of featurequantities of the data for generation 1500.

When it is determined that the number of feature quantities does notmatch the upper limit value, the candidate interpretation factorgeneration unit 1401 returns to step S303 and adds 1 to the value of thevariable n.

When it is determined that the number of feature quantities matches theupper limit value, the candidate interpretation factor generation unit1401 determines whether the processing is completed for all the data forgeneration 1500 (step S311). At this time, when the candidateinterpretation factor generated from the target data for generation 1500does not exist, the candidate interpretation factor generation unit 1401stores a combination of the identification information of the targetdata for generation 1500 and the unclassified flag in the work area.

When it is determined that the processing is not completed for all thedata for generation 1500, the candidate interpretation factor generationunit 1401 returns to step S302 and selects new data for generation 1500.

When it is determined that the processing is completed for all the datafor generation 1500, the candidate interpretation factor generation unit1401 sets the weight of each candidate interpretation factor (stepS312). For example, the following setting method can be considered.

(Setting Method 1) The candidate interpretation factor generation unit1401 sets all weights of the candidate interpretation factors to “1”. Inthis case, “1” is set in a row of the weight 1905 of each entry.

(Setting Method 2) The candidate interpretation factor generation unit1401 sets a weight of a candidate interpretation factor based on aninput of the user. In this case, the value input by the user is set inthe row of the weight 1905 of each entry.

(Setting Method 3) The candidate interpretation factor generation unit1401 calculates the weight of the candidate interpretation factor basedon the combination of interaction values stored in the field of theinteraction value 1908 of the entry corresponding to the candidateinterpretation factor, and sets the weight. In this case, a valuecalculated based on the field of the interaction value 1908 of eachentry is set in the row of the weight 1905 of each entry.

The present invention is not limited to the method of setting theweight.

Next, the candidate interpretation factor generation unit 1401calculates the number of pieces of the second data of each candidateinterpretation factor (step S313). Thereafter, the candidateinterpretation factor generation unit 1401 terminates a series ofprocessing. Specifically, the following processing is executed.

(Processing E1) The candidate interpretation factor generation unit 1401determines whether there exists data for generation 1500 that is notprocessed in the loop processing of the data for generation 1500.

(Processing E2) When the data for generation 1500 that is not processedin the loop processing of the data for generation 1500 exists, thecandidate interpretation factor generation unit 1401 specifies thecorresponding candidate interpretation factor based on the value and thecontribution value of each feature quantity of the unprocessed data forgeneration 1500. The candidate interpretation factor generation unit1401 stores, in the work area, a combination of the identificationinformation of the data for generation 1500 and the identificationinformation (ID 1901) of the specified candidate interpretation factor.The candidate interpretation factor generation unit 1401 updates thefield of the number of pieces of the first data 1906 of each entry ofthe candidate interpretation factor information 1530.

When the corresponding candidate interpretation factor does not exist,the candidate interpretation factor generation unit 1401 stores thecombination of the identification information of the data for generation1500 and the unclassified flag in the work area.

(Processing E3) When data for generation 1500 that is not processed inthe loop processing of the data for generation 1500 does not exist orwhen the processing of (processing E2) is executed, the candidateinterpretation factor generation unit 1401 specifies an interpretationfactor of each data for generation 1500 by referring to the candidateinterpretation factor information 1530. The candidate interpretationfactor generation unit 1401 stores, in the work area, a combination ofthe identification information of the data for generation 1500 and theidentification information (ID 1901) of the specified candidateinterpretation factor.

(Processing E4) The candidate interpretation factor generation unit 1401stores, in the work area, the number of pieces of combinations includingthe unclassified flag as the number of pieces of unclassified data.

(Processing E5) The candidate interpretation factor generation unit 1401specifies a set including only one candidate interpretation factor. Thecandidate interpretation factor generation unit 1401 generates a groupof the specified combination for each type of candidate interpretationfactor. The candidate interpretation factor generation unit 1401 setsthe number of combinations contained in the group in the field of thesecond data 1907 of the entry corresponding to each group by referringto the candidate interpretation factor information 1530.

When the check field 2126 is operated, the candidate interpretationfactor generation unit 1401 deletes the entry whose value in the fieldof the number of pieces of the first data 1906 is smaller than thethreshold before the processing E3 is executed, and executes theprocessing of (processing E2) again.

The candidate interpretation factor generation unit 1401 may calculatethe number of pieces of the second data 1907 based on a value such asthe output value 1705 of the intermediate calculation data 1510 asnecessary. The processing in step S313 has been described above.

FIG. 26 is a flowchart showing processing executed by the conversiontable generation unit 1402 according to the second embodiment.

The conversion table generation unit 1402 generates a candidateconversion table 1540 based on the area division information 1520 andthe candidate interpretation factor information 1530 (step S401).Specifically, the following processing is executed.

(Processing F1) The conversion table generation unit 1402 generates,based on the area division information 1520, a matrix format candidateconversion table 1540 in which the area of the value of the featurequantity is set as a column component and the area of the contributionvalue of the feature quantity is set as a row component. At this timepoint, the candidate interpretation factor data 2000 does not exist inany of the candidate conversion tables 1540.

(Processing F2) The conversion table generation unit 1402 selects atarget entry from the candidate interpretation factor information 1530.

(Processing F3) The conversion table generation unit 1402 selects atarget feature quantity from the feature quantities contained in thetarget entry.

(Processing F4) The conversion table generation unit 1402 specifies acell of the candidate conversion table 1540 corresponding to the targetfeature quantity based on the area (value) 1903 and the area(contribution value) 1904 of the row corresponding to the target featurequantity. The conversion table generation unit 1402 sets the candidateinterpretation factor data 2000 that includes identification informationof the candidate interpretation factor and the value in the field of theweight 1905 of the row corresponding to the target feature quantity inthe specified cell.

(Processing F5) When the processing is not completed for all the featurequantities contained in the target entry, the conversion tablegeneration unit 1402 returns to (processing F3). When the processing iscompleted for all the feature quantities contained in the target entry,the conversion table generation unit 1402 determines whether theprocessing is completed for all the entries of the candidateinterpretation factor information 1530.

(Processing F6) When the processing is not completed for all the entriesof the candidate interpretation factor information 1530, the conversiontable generation unit 1402 returns to (processing F2). When theprocessing is completed for all the entries of the candidateinterpretation factor information 1530, the conversion table generationunit 1402 terminates the processing of step S401. The processing in stepS401 has been described above.

Next, the conversion table generation unit 1402 generates displayinformation for displaying the candidate conversion table 1540, thecandidate interpretation factor information 1530, and the like, andoutputs the display information to the operation receiving unit (stepS402). Accordingly, the confirmation screen 2300 is displayed on theterminal 101. Thereafter, the conversion table generation unit 1402shifts to a waiting state.

When receiving a request via the confirmation screen 2300, theconversion table generation unit 1402 determines whether the receivedrequest is a request of updating information (step S403).

When it is determined that the received request is a request of updatinginformation, the conversion table generation unit 1402 updates thecandidate interpretation factor information 1530 according to thecontent of the request of updating information (step S404). Thereafter,the conversion table generation unit 1402 returns to step S403. In stepS404, for example, the following processing is executed.

When the request of updating information is a deletion request, theconversion table generation unit 1402 searches for an entrycorresponding to the identification information of the candidateinterpretation factor contained in the deletion request from thecandidate interpretation factor information 1530, and deletes the rowcorresponding to the identification information of the feature quantitycontained in the deletion request from the retrieved entry.

When the request of updating information is a modification request, theconversion table generation unit 1402 retrieves an entry correspondingto the identification information of the candidate interpretation factorcontained in the modification request from the candidate interpretationfactor information 1530, and updates the field of the weight 1905 of therow corresponding to the identification information of the featurequantity contained in the modification request of the searched entry.

When the request of updating information is a setting request, theconversion table generation unit 1402 retrieves an entry correspondingto the identification information of the candidate interpretation factorcontained in the setting request from the candidate interpretationfactor information 1530, and sets the interpretation factor in the fieldof the interpretation factor 1909 of the row corresponding to theidentification information of the feature quantity contained in thesetting request of the searched entry.

When the request of updating information is a discarding request, theconversion table generation unit 1402 retrieves an entry correspondingto the identification information of the candidate interpretation factorcontained in the discarding request from the candidate interpretationfactor information 1530, and deletes the entry.

The conversion table generation unit 1402 may update the candidateconversion table 1540 based on the request of updating information.

When the request of updating information is a deletion request, theconversion table generation unit 1402 deletes the candidateinterpretation factor data 2000 corresponding to the identificationinformation of the candidate interpretation factor contained in thedeletion request, from the candidate conversion table 1540 correspondingto the identification information of the feature quantity contained inthe deletion request.

When the request of updating information is a modification request, theconversion table generation unit 1402 retrieves candidate interpretationfactor data 2000 corresponding to the identification information of thecandidate interpretation factor contained in the modification request,from the candidate conversion table 1540 corresponding to theidentification information of the feature quantity contained in themodification request. The conversion table generation unit 1402 updatesthe value of the weight 2002 of the retrieved candidate interpretationfactor data 2000.

When the request of updating information is a setting request, theconversion table generation unit 1402 retrieves the candidateinterpretation factor data 2000 corresponding to the identificationinformation of the candidate interpretation factor contained in themodification request from the candidate conversion table 1540corresponding to the identification information of the feature quantitycontained in the modification request. The conversion table generationunit 1402 replaces the interpretation factor ID 2001 of the retrievedcandidate interpretation factor data 2000 with an interpretation factor.

When the request of updating information is a discarding request, theconversion table generation unit 1402 deletes the candidateinterpretation factor data 2000 corresponding to the identificationinformation of the candidate interpretation factor contained in thediscarding request, from each candidate conversion table 1540. Theprocessing in step S404 has been described above.

When it is determined that the received request is not a request ofupdating information, that is, when it is determined that the receivedrequest is a conversion table output request, the conversion tablegeneration unit 1402 generates the conversion table 500 based on thearea division information 1520 and the candidate interpretation factorinformation 1530, and stores the conversion table 500 generated in theinterpretation factor conversion information 132 (step S405).Thereafter, the conversion table generation unit 1402 terminates theprocessing.

The method of generating the conversion table 500 is similar to that ofthe candidate conversion table 1540. However, in (processing F4), theconversion table generation unit 1402 sets, in the specified cell, theinterpretation factor data 510 including the value in the field of theinterpretation factor 1909 of the target entry and the value in thefield of the weight 1905 of the row corresponding to the target featurequantity.

When the candidate conversion table 1540 is updated based on the requestof updating information in step S404, the candidate conversion table1540 may be output as the conversion table 500 in the interpretationfactor conversion information 132.

The conversion table generation unit 1402 generates the conversion table500, and then transmits a storage request including the conversion table500 to the computer 100-3. The interpretation factor selection unit 130or the result output unit 131 of the computer 100-3 stores theconversion table 500 in the interpretation factor conversion information132.

According to the second embodiment, the computer system can generate theconversion table 500 for each feature quantity from the data forgeneration 1500 without omission, based on an objective criterion. Theinterpretation information 610 generated based on the conversion table500 generated in this way is useful as information indicating the basisof the predicted value of the evaluation target data 300 that isobjective and has no omission.

Third Embodiment

In the third embodiment, a method of generating a candidateinterpretation factor is partially different. Specifically, similarcandidate interpretation factors are integrated. Hereinafter, a thirdembodiment will be described focusing on a difference from the secondembodiment.

The configuration of the computer system according to the thirdembodiment, the hardware configuration and the software configuration ofthe computer 100, and the flow of processing of the computer system arethe same as those in the second embodiment, so that the descriptionsthereof are omitted.

In the third embodiment, a data structure of the candidateinterpretation factor information 1530 is different. FIG. 27 shows anexample of a data structure of the candidate interpretation factorinformation 1530 according to the third embodiment.

For ease of description, the field of the area (value) 1903, the fieldof the area (contribution value) 1904, the field of the weight 1905, thefield of the number of pieces of the first data 1906, the field of thenumber of pieces of the second data 1907, the field of the interactionvalue 1908, and the field of the interpretation factor 1909 are omitted.

An entry stored in the candidate interpretation factor information 1530according to the third embodiment includes a field of integration ID1910 and a field of integration type 1911. The field of integration ID1910 stores identification information set in integrated candidateinterpretation factors. The field of integration type 1911 stores thetype of integrated conditions. In the field of integration type 1911 ofthe entry in which the identification information is set in the field ofintegration ID 1910, either of the terms “normal” and “special” isstored.

FIGS. 28A and 28B are flowcharts showing an example of processingexecuted by the candidate interpretation factor generation unit 1401according to the third embodiment. FIG. 29 shows an example of anintegration condition setting screen 2900 displayed on the terminal 101according to the third embodiment. FIG. 30 shows an example of anintegration confirmation screen 3000 displayed on the terminal 101according to the third embodiment.

The processing from step S301 to step S313 is the same as the processingdescribed in the second embodiment. In the third embodiment, thecandidate interpretation factor generation unit 1401 receives setting ofan integration condition after the processing of step S313 is executed(step S351).

Specifically, the candidate interpretation factor generation unit 1401generates display information for displaying the integration conditionsetting screen 2900, and outputs the display information to theoperation receiving unit. The operation receiving unit transmits thedisplay information to the terminal 101. Here, the integration conditionsetting screen 2900 will be described.

The integration condition setting screen 2900 is a screen presented bythe operation receiving unit, and is displayed on the terminal 101. Theintegration condition setting screen 2900 includes an integrationavailability selection field 2901, an integration condition settingfield 2902, a special integration condition setting field 2903, and asetting button 2904.

The integration availability selection field 2901 is a field forselecting whether to integrate candidate interpretation factors. Thevalue in the integration availability selection field 2901 is output asintegration availability information. The integration availabilityselection field 2901 includes radio buttons 2911 and 2912.

The radio button 2911 is a radio button operated to integrate candidateinterpretation factors. The user operates the radio button 2911 toperform an input to the integration condition setting field 2902 and thespecial integration condition setting field 2903. The radio button 2912is a radio button operated not to integrate the candidate interpretationfactors. When the radio button 2912 is operated, the input to theintegration condition setting field 2902 and the special integrationcondition setting field 2903 is invalidated.

The integration condition setting field 2902 is a field for setting anintegration condition of the candidate interpretation factor. The valuein the integration condition setting field 2902 is output as integrationcondition setting information. The integration condition setting field2902 includes radio buttons 2921, 2922, and 2923.

The radio button 2921 is a radio button operated to set a predefinedintegration condition.

The radio button 2922 is a radio button operated to integrate thecandidate interpretation factors based on the number of pieces of firstdata and the number of pieces of second data. The user operates theradio button 2922 to set the number of pieces of first data or a ratioof the number of pieces of first data to the number of pieces of seconddata.

The radio button 2923 is a radio button operated to integrate thecandidate interpretation factors based on the similarity between thecandidate interpretation factors in distributed representation space.The user operates the radio button 2923 to set a threshold of thesimilarity and a dimension of the distributed representation space. Inthe present embodiment, the candidate interpretation factor generationunit 1401 generates a distributed representation of the data forgeneration 1500 by latent representation of the data for generation 1500based on the dimension of the distributed space, and calculates thesimilarity between the sets of feature quantities constituting eachcandidate interpretation factor (similarity between candidateinterpretation factors) by using the distributed representation. Insteadof the dimension of the distributed representation space, parametersused for other latent representations may be input.

The special integration condition setting field 2903 is a field forsetting an integration condition different from that of the integrationcondition setting field 2902. The value in the special integrationcondition setting field 2903 is output as special integration conditionsetting information. The special integration condition setting field2903 includes a radio button 2931.

The radio button 2931 is a radio button operated to integrate candidateinterpretation factors whose similarity is low or is a negative value inthe integration condition corresponding to the radio button 2923. Thecandidate interpretation factor that satisfies the special integrationcondition information is integrated as a candidate interpretation factorhaving an opposite meaning.

The setting button 2904 is an operation button for setting values set inthe integration availability selection field 2901, the integrationcondition setting field 2902, and the special integration conditionsetting field 2903. When the setting button 2904 is operated, theterminal 101 transmits a setting request including the integrationavailability information, the integration condition setting information,and the special integration condition setting information.

The integration condition setting screen 2900 may include an operationbutton for saving, in the computer 100-2, the integrated conditionsetting information, the integration condition setting information, andthe special integration condition setting information, which are set viathe integration condition setting screen 2900, and an operation buttonfor instructing the destruction or initialization of the information.

The integration condition setting screen 2900 has been described above.The description returns to FIG. 28B.

When receiving the setting request from the operation receiving unit,the candidate interpretation factor generation unit 1401 executesintegration processing based on the integration availabilityinformation, the integration condition setting information, and thespecial integration condition setting information contained in thesetting request (step S352). Thereafter, the candidate interpretationfactor generation unit 1401 terminates the processing. Specifically, thefollowing processing is executed.

(Processing G1) The candidate interpretation factor generation unit 1401determines whether the integration availability information includes avalue corresponding to the radio button 2912. When the integrationavailability information includes a value corresponding to the radiobutton 2912, the candidate interpretation factor generation unit 1401terminates the integration processing without integrating the candidateinterpretation factors.

(Processing G2) When the integration availability information includes avalue corresponding to the radio button 2911, the candidateinterpretation factor generation unit 1401 calculates an index used tointegrate candidate interpretation factors based on the integrationcondition setting information.

When the integration condition setting information includes a valuecorresponding to the radio button 2922, the candidate interpretationfactor generation unit 1401 calculates the number of pieces of firstdata of each candidate interpretation factor or a ratio of the number ofpieces of first data to the number of pieces of second data. When theintegration condition setting information includes a value correspondingto the radio button 2923, the candidate interpretation factor generationunit 1401 calculates the similarity between the candidate interpretationfactors.

(Processing G3) The candidate interpretation factor generation unit 1401specifies, based on a comparison result between the calculated index andthe threshold, a set of candidate interpretation factors that can beintegrated. In addition, the candidate interpretation factor generationunit 1401 specifies, based on the calculated index and the specialintegration condition setting information, a set of candidateinterpretation factors that can be integrated. The candidateinterpretation factor generation unit 1401 stores, in a work area,information of a set of specified interpretation factors.

(Processing G4) The candidate interpretation factor generation unit 1401generates display information for presenting the set of specifiedcandidate interpretation factors, and outputs the generated displayinformation to the operation receiving unit. Accordingly, theintegration confirmation screen 3000 is displayed on the terminal 101.The candidate interpretation factor generation unit 1401 integrates aset of candidate interpretation factors based on an operation via theintegration confirmation screen 3000. Here, the integration confirmationscreen 3000 will be described.

The integration confirmation screen 3000 is a screen presented by theoperation receiving unit, and is displayed on the terminal 101. Theintegration confirmation screen 3000 includes an integration targetdisplay field 3001, a candidate interpretation factor detail displayfield 3002, an integration button 3003, a discarding button 3004, a nextbutton 3005, and a completion button 3006.

The integration target display field 3001 is a field for displayinginformation on the specified candidate interpretation factor and adetermination basis. The integration target display field 3001 includesa candidate interpretation factor display field 3011 and an indexdisplay field 3012.

The candidate interpretation factor display field 3011 is a field fordisplaying identification information of the specified candidateinterpretation factor. The index display field 3012 is a field fordisplaying an index used during determination.

The candidate interpretation factor detail display field 3002 displaysdetailed data of the specified candidate interpretation factor.

The integration button 3003 is a button operated to integrate a set ofcandidate interpretation factors displayed on the integrationconfirmation screen 3000. When the integration button 3003 is operated,the terminal 101 transmits an integration request including informationof the set of candidate interpretation factors.

When receiving the integration request, the candidate interpretationfactor generation unit 1401 specifies an entry group to be integratedfrom the candidate interpretation factor information 1530 based on theinformation of the set of candidate interpretation factors contained inthe integration request. The candidate interpretation factor generationunit 1401 sets the same identification information as that of the fieldof integration ID 1910 of the specified entry group, and sets either ofthe terms “normal” and “special” in the field of integration type 1911.The term “normal” is set in the field of integration type 1911 when thecandidate interpretation factors are integrated based on the integrationcondition setting information. The term “special” is set in the field ofintegration type 1911 when the candidate interpretation factors areintegrated based on the special integration condition settinginformation.

The discarding button 3004 is a button operated not to integrate the setof candidate interpretation factors displayed on the integrationconfirmation screen 3000. When the discarding button 3004 is operated,the terminal 101 transmits a discarding request including information ofthe set of candidate interpretation factors.

When receiving the discarding request, the candidate interpretationfactor generation unit 1401 deletes the information of the set ofcandidate interpretation factors contained in the discarding requestfrom the work area.

The next button 3005 is a button operated to display information of anext set of interpretation factors.

The completion button 3006 is a button operated to terminate theoperation via the integration confirmation screen 3000. When thecompletion button 3006 is operated, the terminal 101 transmits acompletion notification.

The integration confirmation screen 3000 has been described above. Theintegration processing will be described again.

(Processing G5) When receiving the completion notification, thecandidate interpretation factor generation unit 1401 terminates theintegration processing. The processing in step S352 has been describedabove.

The candidate interpretation factor generation unit 1401 mayautomatically integrate the set of specified candidate interpretationfactors without displaying the integration confirmation screen 3000.

In the processing of generating the conversion table 500 (FIG. 26), theconversion table generation unit 1402 sets the value set in the field ofintegration ID 1910 in the interpretation factor ID 2001 of thecandidate interpretation factor data 2000 corresponding to theintegrated candidate interpretation factor. The conversion tablegeneration unit 1402 sets the same interpretation factor for thecandidate interpretation factor whose identification information is setin the field of integration ID 1910.

According to the third embodiment, the computer system can avoidgeneration of duplicate or similar interpretation factors by integratinga plurality of candidate interpretation factors. In addition, thecalculation efficiency can be improved.

The invention is not limited to the above-described embodiment, andincludes various modifications. In addition, for example, theabove-described embodiment has been described in detail for easyunderstanding of the invention, and the invention is not necessarilylimited to those including all the configurations described above. Inaddition, a part of the configuration of each embodiment can be added,deleted, or replaced with another configuration.

Each of the configurations, functions, processing units, processingmethods described above may be partially or entirely implemented byhardware such as by designing with an integrated circuit. Further, theinvention can also be implemented by program code of software thatimplements the functions of the embodiments. In this case, a storagemedium storing the program code is provided to a computer, and aprocessor provided in the computer reads out the program code stored inthe storage medium. In this case, the program code itself read out fromthe storage medium implements the functions of the embodiments describedabove, and the program code itself and the storage medium storing theprogram code constitute the invention. Examples of the storage mediumfor supplying such a program code include a flexible disk, a CD-ROM, aDVD-ROM, a hard disk, a solid state drive (SSD), an optical disk, amagneto-optical disk, a CD-R, a magnetic tape, a nonvolatile memorycard, and a ROM.

For example, the program code that implements the function described inthe present embodiment can be implemented by a wide range of programs orscripting languages, such as an assembler, C/C++, perl, Shell, PHP, andJava (registered trademark).

Further, the program code of the software implementing the functions ofthe embodiments is distributed via a network, so that the program codeis stored in a storage unit such as the hard disk or the memory of thecomputer or in a storage medium such as a CD-RW or the CD-R. Theprocessor provided in the computer may read out and execute the programcode stored in the storage unit or the storage medium.

In the embodiment described above, the control lines and the informationlines are considered to be necessary for description, and all controllines and information lines are not necessarily shown in the product.All configurations may be connected to each other.

What is claimed is:
 1. A computer system comprising: at least onecomputer that includes a processor and a memory connected to theprocessor, wherein the memory stores: model information for predictingan event of a target based on input data including a value of aplurality of feature quantities indicating a state of the target, andinterpretation factor conversion information for managing aninterpretation factor interpreting a basis of a prediction result forthe input data, the interpretation factor being determined by a value ofeach of the plurality of feature quantities contained in the input dataand a first evaluation value of the value of each of the plurality offeature quantities contained in the input data, the first evaluationvalue showing magnitude of contribution of the value of the featurequantity to a prediction result for the input data, and the processor isconfigured to: calculate, when evaluation target data which is the inputdata to be evaluated is input, a prediction result for the evaluationtarget data based on the model information, calculate a first evaluationvalue of each of the plurality of feature quantities contained in theevaluation target data, specify a corresponding interpretation factor,based on the value and first evaluation value of each of the pluralityof feature quantities contained in the evaluation target data, byreferring to the interpretation factor conversion information, andgenerate display information for presenting the specified interpretationfactor and output the display information.
 2. The computer systemaccording to claim 1, wherein the interpretation factor conversioninformation includes conversion data in which a set is associated withthe interpretation factor, the set including, as elements, a combinationof an area of the value of the feature quantity contained in the inputdata and an area of the first evaluation value of the feature quantitycontained in the input data, and the processor is configured to: specifyan area including a value of each of the plurality of feature quantitiescontained in the evaluation target data, specify an area including afirst evaluation value of each of the plurality of feature quantitiescontained in the evaluation target data, retrieve, by referring to theinterpretation factor conversion information, the conversion datacorresponding to a subset, the subset including, as elements, acombination of the area including the value of each of the plurality offeature quantities contained in the evaluation target data and the areaincluding the first evaluation value of each of the plurality of featurequantities contained in the evaluation target data, and specify theinterpretation factor from the retrieved conversion data.
 3. Thecomputer system according to claim 2, wherein the processor isconfigured to: acquire area division information defining areas of thevalue and the first evaluation value of each of the plurality of featurequantities contained in the input data, acquire data for generationwhich is input data for generating the interpretation factor conversioninformation, calculate a prediction result of the data for generationand the first evaluation value of each of the plurality of featurequantities contained in the data for generation, generate subsets of theplurality of feature quantities contained in the input data, select atarget subset from the subsets, calculate a second evaluation valueshowing magnitude of contribution of a combination of values of aplurality of feature quantities to a prediction result for the data forgeneration, the plurality of feature quantities being contained in thedata for generation and being elements of the target subset, specify,when the second evaluation value is larger than a threshold, an areaincluding the value of the feature quantity, based on the area divisioninformation, for the value of each of the plurality of featurequantities that are contained in the data for generation and areelements of the target subset, and specify an area including the firstevaluation value of the feature quantity, based on the area divisioninformation, for the first evaluation value of each of the plurality offeature quantities that are contained in the data for generation and arecontained in the target subset, generate a set as candidate conversiondata, the set including the specified area of the values of theplurality of feature quantities and the specified area of the firstevaluation values as elements, and generate the conversion data byassociating the interpretation factor with the candidate conversion dataand store the conversion data in the interpretation factor conversioninformation.
 4. The computer system according to claim 3, wherein theprocessor is configured to: acquire integration condition informationdefining a condition under which a plurality of pieces of the candidateconversion data is integrated, specify, based on the integrationcondition information, the plurality of pieces of candidate conversiondata which are able to be integrated, integrate the plurality of piecesof specified candidate conversion data, and generate the conversion databy associating the interpretation factor with the plurality of pieces ofintegrated candidate conversion data.
 5. The computer system accordingto claim 4, wherein the integration condition information includes atype and a threshold of an index indicating similarity between thecandidate conversion data, and the processor is configured to: calculatethe index between the candidate conversion data, and specify, based on acomparison result of the index and the threshold, the plurality ofpieces of candidate conversion data which are able to be integrated. 6.A method of presenting information on a prediction result of predictionfor input data executed by a computer system, wherein: the computersystem includes at least one computer that includes a processor and amemory connected to the processor, the memory storing model informationfor predicting an event of a target based on input data including aplurality of feature quantities indicating a state of the target, andinterpretation factor conversion information for managing aninterpretation factor interpreting a basis of a prediction result forthe input data, the interpretation factor being determined by a value ofeach of the plurality of feature quantities contained in the input dataand a first evaluation value of the value of each of the plurality offeature quantities contained in the input data, the first evaluationvalue showing magnitude of contribution of the value of the featurequantity to a prediction result for the input data, the method ofpresenting information comprising: a first step in which the processoris configured to calculate, when an evaluation target data which is theinput data to be evaluated is input, a prediction result for theevaluation target data based on the model information; a second step inwhich the processor is configured to calculate a first evaluation valueof each of the plurality of feature quantities contained in theevaluation target data; a third step in which the processor isconfigured to specify a corresponding interpretation factor, based onthe value and first evaluation value of each of the plurality of featurequantities contained in the evaluation target data, by referring to theinterpretation factor conversion information; and a fourth step in whichthe processor is configured to generate display information forpresenting the specified interpretation factor and output the displayinformation.
 7. The method of presenting information according to claim6, wherein the interpretation factor conversion information includesconversion data in which a set is associated with the interpretationfactor, the set including, as elements, a combination of an area of thevalue of the feature quantity contained in the input data and an area ofthe first evaluation value of the feature quantity contained in theinput data, wherein the third step includes: a step in which theprocessor is configured to specify an area including a value of each ofthe plurality of feature quantities contained in the evaluation targetdata; a step in which the processor is configured to specify an areaincluding a first evaluation value of each of the plurality of featurequantities contained in the evaluation target data; a step in which theprocessor is configured to retrieve, by referring to the interpretationfactor conversion information, the conversion data corresponding to asubset, the subset including, as elements, a combination of the areaincluding the value of each of the plurality of feature quantitiescontained in the evaluation target data and the area including the firstevaluation value of each of the plurality of feature quantitiescontained in the evaluation target data; and a step in which theprocessor is configured to specify the interpretation factor from theretrieved conversion data.
 8. The method of presenting informationaccording to claim 7, further comprising: a fifth step in which theprocessor is configured to acquire area division information definingareas of the value and the first evaluation value of each of theplurality of feature quantities contained in the input data; a sixthstep in which the processor is configured to acquire data for generationwhich is input data for generating the interpretation factor conversioninformation; a seventh step in which the processor is configured tocalculate a prediction result of the data for generation and a firstevaluation value of each of a plurality of feature quantities containedin the data for generation; an eighth step in which the processor isconfigured to generate subsets of the plurality of feature quantitiescontained in the input data; a ninth step in which the processor isconfigured to select a target subset from the subsets; a tenth step inwhich the processor is configured to calculate a second evaluation valueshowing magnitude of contribution of a combination of values of aplurality of feature quantities to a prediction result for the data forgeneration, the plurality of feature quantities being contained in thedata for generation and being elements of the target subset; an eleventhstep in which the processor is configured to specify, when the secondevaluation value is larger than a threshold, an area including the valueof the feature quantity, based on the area division information, for thevalue of each of the plurality of feature quantities that are containedin the data for generation and are elements of the target subset, andspecify an area including the first evaluation value of the featurequantity, based on the area division information, for the firstevaluation value of each of the plurality of feature quantities that arecontained in the data for generation and are contained in the targetsubset; a twelfth step in which the processor is configured to generatea set as candidate conversion data, the set including the specified areaof the values of the plurality of feature quantities and the specifiedarea of the first evaluation values as elements; and a thirteenth stepin which the processor is configured to generate the conversion data byassociating the interpretation factor with the candidate conversion dataand store the conversion data in the interpretation factor conversioninformation.
 9. The method of presenting information according to claim8, wherein the twelfth step includes: a step in which the processor isconfigured to acquire integration condition information defining acondition under which a plurality of pieces of the candidate conversiondata is integrated; a step in which the processor is configured tospecify, based on the integration condition information, the pluralityof pieces of candidate conversion data which are able to be integrated;and a step in which the processor is configured to integrate theplurality of pieces of specified candidate conversion data, thethirteenth step includes a step in which the processor is configured togenerate the conversion data by associating the interpretation factorwith the plurality of pieces of integrated candidate conversion data.10. The method of presenting information according to claim 9, whereinthe integration condition information includes a type and a threshold ofan index indicating similarity between the candidate conversion data,and the method of presenting information further comprising: a step inwhich the processor is configured to calculate the index between thecandidate conversion data; and a step in which the processor isconfigured to specify, based on a comparison result of the index and thethreshold, the plurality of pieces of candidate conversion data whichare able to be integrated.